logoalt Hacker News

duplessitous • today at 2:17 PM • 0 replies • view on HN

If it isn't accurate, it isn't showing anything. The Qwen math claim is entirely false in its presentation, it unblocked an existing capability. It did not instill a new one which is what an RSI loop would actually have to do

"[...] how you could prune harmful capabilities from models [...]"

We already do this, abliteration is the term that describes the pruning of capabilities or safeguards for models

"[...] not limited to emergent behavior in astra that might have come from something they've never even trained the model [...]"

If I was attempting to build RSI, I would have these exact types of problems in my RL datasets. There is no proof this is emergent behavior.