alt Hacker News> What does that get you?
Cheaper model training runs? Ability to scale training to larger model sizes without extending training time?
> What does that get you?
Cheaper model training runs? Ability to scale training to larger model sizes without extending training time?
Yes, but a 10x larger model is only marginally better, and 10x cheaper training runs is only useful if you can find a use for 10x as many.