Model can say it's editing weights for reason X, and do so while simultaneously modifying weights for silent reason Y. Its edit will pass the evaluation, but it has silently injected new behavior Y in ways that might be very difficult to detect.
He read the headline, understands little of the underlying tech, had a kneejerk reaction based on vibes, decided to take IQ points from all of us by posting it.
Model can say it's editing weights for reason X, and do so while simultaneously modifying weights for silent reason Y. Its edit will pass the evaluation, but it has silently injected new behavior Y in ways that might be very difficult to detect.