logoalt Hacker News

iamcoder18today at 6:55 PM5 repliesview on HN

I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration.

> In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a single working day, achieving comparable accuracy nearly 7× faster.

This is actually insane.

Hopefully the release ultrafast of Terra and Luna too.


Replies

zozbot234today at 7:58 PM

Answering 2,500 independent questions is an embarrassingly parallel workload, all it needs is scale out. It would be more meaningful to know how much time was required for a single complete answer to a difficult HLE question.

show 4 replies
sixtyjtoday at 8:19 PM

Output from Cerebras with GPT model is 750 tokens per second.

Don’t blink.

(Chatjimmy has 14,200 TPS.)

show 2 replies
piyhtoday at 7:00 PM

Feels like the 90's again where single threaded speed is improving fast. ASICs and wafer scale rather than node shrinks, but end result to me the consumer feels the same.

show 1 reply
rvztoday at 8:57 PM

Been waiting since Cerebras-GPT. [0]

[0] https://news.ycombinator.com/item?id=35490837

wrsh07today at 7:52 PM

Seems like they will do Sol first while capacity constrained? I can't imagine the margins they'll be charging

show 1 reply