If you only outdo humans 50% of the time you're never going to get consensus on if you've qualified. Whereas outdoing 90% of humans on 90% of all the most difficult tasks we could come up with is going to be difficult to argue against.

This benchmark is only one such task. After this one there's still the rest of that 90% to go.

Beating humans isn't anywhere near sufficient to qualify as ASI. That's an entirely different league with criteria that are even more vague.

I’d be hesitant to call that ASI if it’s pretty obvious how you’d write a regular old program to solve it.

People are still debating whether these models exhibit any kind of intelligence and any kind of thinking. Setting the bar higher then necessary is welcome, but at this point I’m pretty sure everyone’s opinions are set in stone.

There's a single true definition of AGI, open the page about AGI on Wikipedia but using archive.org on a snapshot from 10 years ago.

All the rest is bullshit made up by LLM labs to make it seem like they hit AGI by dumbing down its definition.

https://web.archive.org/web/20150108000749/https://en.wikipe...

In retrospect, it seems obvious that we hit AGI by a reasonable "at least as intelligent as some humans" definition when o3 came out, and everything since then has been goalpost moving by people who have higher and higher bars for which percentile human they would be willing to employ (or consider intellectually capable). People should really just use the term "ASI" when their definition of AGI excludes the majority of humans.

Edit: Here's the guy who coined the term saying we're already there. Everything else is arguing over definitions.

https://x.com/mgubrud/status/2036262415634153624

> Well, Lars, I INVENTED THE TERM and I say we have achieved AGI. Current models perform at roughly high-human level in command of language and general knowledge, but work thousands of times faster than us. Still some major deficiencies remain but they're falling fast.