alt Hacker NewsThese tests are pointless when often models get nerfed few days after release. Astra today is way dumber than just few days ago.
Replies
cbg0 • today at 6:30 PM
Not Astra but Sol hasn't been nerfed: https://marginlab.ai/trackers/codex/
➕ show 1 reply
Is that something we have credible evidence for? Do they serve a better model when artificialanalysis (the benchmark site) is making the requests, and so on?