The 'Astra can read and write weights to improve Qwen' post is misleading

[flagged]

[dead]

[dead]

[dead]

[dead]

[flagged]

> The “just by looking at the raw weights” setting (raw floats in context, no tools, no running the network) only appears in experiments 1, 2 and 4, on networks of 17K–172K weights.

The author is being equally disingenuous.

That piece by itself is insightful. Meaning, LLMs are capable of deciphering what a model can do by looking at the weights alone.

The fact that we can’t squeeze enough of them through the context is a technical limitation. However, this does not diminish the importance of this finding/discovery. This is an unexpected emergent behaviour.

A tangent, but I’m now realizing that the “recursive self improvement” thing that these LLM companies are talking about is (probably?) not about unsupervised Claude Code instances improving the training process, but rather LLMs modifying their own weights through tooling directly. It would certainly open up a whole new process for model optimization.

The blog post exhibits the same flawed logic that many academic reviews do — looking for reasons to reject a work rather than accept it. It comes off cynical and contrarian. It's easy to find issues with an academic paper, but ACL has consistently pushed for a higher standard [1].

Being the author of spaCy, he should know better. I certainly enjoyed the straightforward simplicity of spaCy when I used it in my research (though I haven't touched it in years).

[1]: https://aclrollingreview.org/reviewerguidelines#-task-3-writ...