alt Hacker NewsWhat's with the deceptive graph on top? Not including Astra can't have been an oversight, did the model compare poorly to it?
Replies
scottyah • today at 4:32 PM
Deceptive? An extremely quick google search would answer your question. OpenAI pulled out of Cursor before they released Astra so it never got that benchmark.
➕ show 1 reply
Iolaum • today at 4:21 PM
I wonder if that means that SpaceX evals show that they consider astra better than fable or that they hate Sam&co so much they don't want to show their stuff.
➕ show 3 replies
https://openai.com/index/our-decision-on-cursor-following-it...
This explains why. Mentioned in another comment, but cursorbench explicitly tests with Cursor as the harness, and OpenAI doesn't allow them to use Astra in Cursor.