I even wonder if the frontier AI models are really as capable as they claim or if the companies behind them have just special cases all the “hard” questions.

For example, the earlier generative LLMs couldn’t correctly answer ‘how many r’s in “strawberry”?’ due to the underlying nature of the tokens.

If they get it correct today, how do they do it? It feels like we’re being deceived by the Wizard of Oz…