alt Hacker NewsMiMo RL'd their latest models for $3.5M (million, not billion). It's hardware, software, and algos
Also look to the history of stable diffusion
Replies
lifeisloving • today at 12:03 AM
Training costs are not in free fall, and not every "post training" process is equal, Everyone does this differently. The costs certainly haven't gone down, to train a new 10T param model from scratch, like at all.
You cannot point at Xiaomis tiny model and compare it to training a 10T param model. The levels of dunning krueger are getting out of hand.
You also arent calculating the salaries of the people involved, of which there's only a few thousand worldwide in your hiring pool.
➕ show 2 replies
I believe that’s post-training, not pre-training.