alt Hacker NewsThere’s nothing much new about the architecture. The real gains come from the usage traces.
It turns out that having a text based interface for a text-trained model creates a very nice feedback loop.
Right now as we speak, people are generating text traces on anthropic and OpenAI servers that teach their models to do everything under the sun, text wise.
So people right now getting super mad at how dumb the model is when reverse-engineering a super complex function from binary, when they write “stop, you dumb robot, you are going wrong, go this way thank you very much” are actually leaving a lesson in the form of the "chat" text history.
Some may say that each bad word get us closer to ASI.
That and obviously the order of magnitude more efficient GPUS we got that allow for different tradeoffs at training time.
Replies
> There’s nothing much new about the architecture. The real gains come from the usage traces.
sorry. how do you know. i am so curious about where exactly gains are coming from but so hard to even get a little bit of insight.
i wish govt would fund these labs and make it free and opensource. way better investment than stupid overseas wars.
Makes me wonder, as people grow to trust the AI more and more, not reading the code and barely skimming the implementation plans and simply rerolling if something doesn't work, will the value of these chats erode? Thinking back 1-1.5 years I was closely monitoring what these agents did and steering them quite aggressively. These days not so much. Where will RL signals come from when it approaches humans capabilities ever closer? How well does self play work for coding work? What about multistep tasks where it isn't just about being good at a single task, but evolving a codebase over time in the face of changing requirements?