alt Hacker NewsThe tech is at the point where synthetic data is almost high enough quality to train further models on, AND there are so many human users of cloud AI, that their usage can be gathered and trained upon (it's human generated and used in real world situations, so, it is of a good enough quality). The cat is fully out of the bag.
This deals with half of one of the three scaling factors only. Presumably data production (choosing your training data, etc), model size, and training time/power still contribute huge costs. Who's investing that cash if open-source models will distill out enough to compete without the investment costs?