logoalt Hacker News

aesthesia • today at 5:21 PM • 5 replies • view on HN

It's worth noting that the only modification the model made to improve itself was to reduce the length of its reasoning traces to avoid hitting the token limit. This is basically the same as the result from the recent weight-space programming article (https://rikiyat.github.io/blog/weight-space-programming/), which found that even Astra wasn't able to improve the performance of a model without this kind of obvious flaw.


Replies

tb99 • today at 6:52 PM

It's also interesting to look at how the agent does the fix. Appendix E5 shows what it's doing:

- Find a common category of error (too many tokens).

- Calculate aggregated difference between states in examples in that error category and other examples.

- Update weights to push states along that difference direction toward the non-error states.

In other words, deprived of gradient descent as a tool, it approximates it. It's clever, and it's a neat experiment. It doesn't show any superhuman insight into its weights -- with the same constraints, a human would likely take a similar approach. But it's early days for self modification and it will be interesting to see if models get a deeper understanding of weights as they get smarter.

➕ show 1 reply
gopalv • today at 6:18 PM

The capability part is just a glimmer of what's possible.

I've been playing with very small models with Astra and it does look like the model can see things in the weights that I cannot see into.

The "fix it" loop is no longer RL in the traditional gradient descent with backprop, it can go and make targeted fixes to the weights directly instead of rerunning with new training data.

The weight collapsing loop which resulted in weight patches on top of a "super tiny model" - https://demos.isotopes.ai/pages/weighing-machine/

This makes for a "Time machine" for models easily

haickernews • today at 6:04 PM

Yeah, but it's likely you can RLVR on this capability. Extrapolate from there.

➕ show 1 reply
jcims • today at 5:51 PM

>even Astra wasn't able to improve the performance of a model without this kind of obvious flaw

Where are you getting this conclusion from that paper? It's both able to deduce what's happening from looking at the weights and write new weights from scratch just from a specification.

➕ show 3 replies
E-Reverance • today at 5:54 PM

This is different paper

➕ show 1 reply