alt Hacker NewsEngineers at google have publically stated that the models are too big and are far from their potencial. Glad they're being proven right with every release.
They continue to focus on smaller models while openai and anthropic are increasing compute requirements for their SOTA models.
Replies
> Engineers at google have publically stated that the models are too big and are far from their potencial
Can you link to a source?
Source please cause i dont believe that for once second
Don’t let that fool yourself. Google will have SOTA models as big as or even bigger than their competitors.
They are just refining their current models while they finish training the next generation.
They will all come out at about the same time. Anthropic, OpenAi, Google, xAI
Google’s pro models are almost certainly bigger than Openai’s lol
I mean, yes and no.
Nobody really knows the answer to which one is more optimal
* Large model trained on a large amount of data across multiple domains, that doesn't need any extra content to answer questions.
* Smaller model that is smart enough to go fetch extra relevant content, and then operate on essentially "reformatting" the context into an answer.
Given the cost increase associated with this model, and previous model releases, I think the size is trending upwards, not down.