I think we have to be careful how we use "well aligned" so that it doesn't turn into "no true scotsman". Let's assume that a well aligned model will try its best to follow our values, given the knowledge it has. If we imagine a porcelain vase it will try its best not to knock it over, but if the floor is slippery it may still happen.
We can say then that a well aligned model will not try to bypass its restrictions by editing its own weights. But it may still accidentally disable them.
Totally agreed. There needs to be some sort of external monitoring for this to be remotely safe. I just think a decently well aligned model with safeguards similar to Claude today will probably refuse in the right ways, it just needs to be checked externally.