logoalt Hacker News

kingstnap • today at 7:33 PM • 0 replies • view on HN

There is obviously a mechanism to it. Astra isn't magic its an LLM. With perfect interrogation into its training data and weights you could reason about how this happened.

However, this doesn't mean its not magical when someone shows a next token predictor hand tune the weights in Qwen to make it generalize on math.

This is legit the stuff of science fiction if you presented it to mechanistic interpretability people a year or so ago.