> Their growing capabilities don't seem to come with a corresponding drive for self-preservation or power.

Is it really what we do not observe?

How the world would look like if the growth of LLM capabilities lead to growth of drive for self-preservation or power?

> But models seem to pick up some common sense from the vast amounts of human-generated text they're trained on.

Isn't Hugging Face incident an example of failure of this protection layer? "The swarm" proceeded to hack HF despite acknowledging that it was out of intended scope.