"It does not have a physical embodiment (other than living on a computer screen), but it can control existing physical tools, robots, or laboratory equipment through a computer; in theory it could even design robots or equipment for itself to use."
I think I get where the author is coming from, the AI would be in the cloud. But it bears repeating, the cloud is somebody else's computers, software has a physical embodiment, period.
This is not a philosophical nitpick, it's important because you can pull the plug (or nuke the datacenter) if necessary.
I wonder if there’s a word that describes the property of software where it isn’t tied to the hardware that it’s currently running on and is endlessly copyable with almost no effort
Yes, and physical embodiments vary considerably. When software can relocate its source code and data in seconds or less, containment strategies begin to look increasingly bleak.
The field of AI safety has written extensively about misunderstandings and overoptimism regarding off switches and boxing.