There's no question that training leading LLMs requires some serious expertise and know-how, but surely already having advanced LLMs/agents must be helping tremendously not only for software engineers but also for those working on LLMs themselves.

I also think we're seeing the sigmoid approaching.

I think the moat is going to be compute. So far compute needed to push the frontier is still extremely cheap so the capital can afford to spread its bets. But when further improvement is going to cost in trillions, capital will have to pick a winner and bet only on him. It won't be a matter of finding the best bet, it will be a matter of survival.

This will cause the picked winner to get massively ahead with sheer compute alone used both for training and inference dedicated to recursive self improvement.