logoalt Hacker News

jrflo • today at 5:49 PM • 0 replies • view on HN

Yes, they started to hit a data wall, but I thought that much of the progress of the last couple years came from reinforcement learning after pretraining, which is mostly building complex environments for the models to reason in, rather than accumulating piles of raw text.