logoalt Hacker News

thefourthchime • yesterday at 7:07 PM • 3 replies • view on HN

Pac-Man Bench:

Considering the price, no model comes close to being as good as this. However, it did take an extremely long time.

TIME 19m COST $0.16 https://jonclegg.github.io/pacman-bakeoff/#claude-haiku-5-5

All results: https://jonclegg.github.io/pacman-bakeoff/

Replies

myzie • yesterday at 7:31 PM

Interesting that you have gpt-6-luna at $0.01 vs. claude-haiku-5-5 at $0.16 for this task. I see the score disparity though and I played them briefly. My takeaway from this is that the choice between Luna and Haiku 5.5 may remain nuanced. Luna may be a lot cheaper still and good enough for some jobs. Is that your read of the results?

➕ show 1 reply
rpcope1 • yesterday at 7:43 PM

Something is not right there. DSv4.1 flash shows $1.89 for tens of thousands of tokens? What am I missing?

onlyrealcuzzo • yesterday at 7:14 PM

How have you avoided being sued by Namco?

➕ show 1 reply