alt Hacker NewsIn this podcast episode[0] he does talk about this kind of model and how it "learns about physics" through experience instead of just ingesting theorical material.
It's quite eye opening.
In this podcast episode[0] he does talk about this kind of model and how it "learns about physics" through experience instead of just ingesting theorical material.
It's quite eye opening.
The way I see it, the "world models" he wants to train require a magnitude more compute than what LLM training requires since physical data is likely much more unstructured than internet data.
He raised $1b but that seems way too little to buy enough compute to train.
My bet is that OpenAI or Anthropic or both will eventually train the model that he always wanted because they will use revenue from LLMs to train a world model.