logoalt Hacker News

woah • today at 6:11 PM • 1 reply • view on HN

The recent crop of weight rewriting papers are more valuable for their inspirational content than their current abilities. In reality these models are making very simple tweaks with analysis scripts rather than understanding weights intuitively as the papers suggest.

However, once LLMs are pre-trained with huge quantities of serialized LLM weights and corresponding reasoning traces, this may change.


Replies

mlmonkey • today at 6:38 PM

Exactly. If I had access to a boatload of GPUs, that's what I'd be doing. Maybe the frontier labs are already doing it (they employ way smarter people than me, of course).