alt Hacker NewsIronically, I think he was right! In fact some of his initial experiments, like copycat, were about predicting patterns. You can see next-token-prediction from there. But I think he always held out for an algorithmic/logical method rather than a purely statistical one.
If he had accepted the "Bitter Lesson", I think he would have been at the forefront of LLMs.
Maybe you haven't noticed that the "Bitter Lesson" had itself a "Bitter Lesson" - that scaling pure data and compute did not lead to AGI: diminishing training returns, GPT-5 disappointment, even openAI stating it was the last 'pure scale' model.
The path forward all big llm providers ("ai" labs) have gone is neuro-symbolic (even though they publicly would never labeled it as such to not admit critics like Gary Marcus were right - even though all their actions actually point in that direction).