alt Hacker NewsYeah giving a 1 hour limit for code generation makes the whole thing a bit incoherent. Plus, performance is just going to depend on how closely the models have memorized the best open-source Starcraft bots and can reproduce them.
Replies
boxed • today at 5:29 PM
> Plus, performance is just going to depend on how closely the models have memorized the best open-source Starcraft bots and can reproduce them.
If that's the case then these models are failing extremely hard. 7% win rate against human written AIs that they should be copying from memory? A copy paste command should do 50%!
If Astra is leading the pack...Considering it's supposed to be one of the slowest model.