alt Hacker NewsA large base model is ~90% of the effort/cost to making a frontier model, and by itself is not useful for much. There aren't many controversial decisions to make when training a base model - you mostly just train a model as big as possible on as much human-generated data as possible.
But it makes it far easier for other parties to do that last 10% to train it to be useful.
However, there are many choices one makes in the last 10% of training which will impact the models behaviour when it comes to AI safety, political opinions, willingness to help make a bomb, etc etc.
> A large base model is ~90% of the effort/cost to making a frontier model
This kind of figures is often quoted but where does that estimation comes from? Isn't that legacy number from the era when all the post training WS just instruction following and RLHF? I'd be surprised if current generation if reinforcement learning was still a few percent of the training cost.