logoalt Hacker News

2ndorderthoughtyesterday at 11:09 PM1 replyview on HN

I really don't think open models will lose. I think they are cheaper to train because they have to be more efficient than the monstrosities we have now.

There is no theory that says the current frontier models cannot exist in models with 1/100th the compute waste ;). When we start trending in that direction, and oh wow we truly are, there will be no reason for these services. You could run them on your own hardware without serious investments.

The moat openai and anthropic have is them among others have attempted to buy all of the computer hardware for the next two years. That's intentional. They know the only existential threat to them is anyone coming up with a way to do this better than them. It's already happened and it's going to become more and more divergent.


Replies

drivebyhootingtoday at 1:34 AM

I’m interested in learning more about your theory that these models can be trained more cheaply. Is anyone doing it from scratch, rather than adversarial distillation?

show 2 replies