alt Hacker NewsI agree, i don't think it is the core problem.
Meta doesn't seem to be able to produce anything close to a frontier model. The selling of compute capacity seems to be acceptance of "compute is wasted on this crappy avocado model, we'd be better off allowing something better to run".
The problem is clearly in the model architecture, the training and the data fed into the model which is causing them to give up on using their compute exclusively for their own models. They can't get it right so may as well sell the compute to someone that can.
Replies
Meta has made some very strange decisions in terms of who it's hired to lead various aspects of AI, including the model-building efforts. Also lots to marvel at re: its ability to coordinate (or not coordinate) various efforts by all these big brains.
Can't help but think that Meta's digital networking expertise is built atop a human-networking clusterf*ck
Does meta have the research talent to create a SOTA frontier model? Yann LeCun has left Meta and I don’t think either alexandr wang or zuck have enough credibility to attract talent to create one.
If their training base is dominated by Facebook and Instagram posts then it makes sense that their model is full of shit.