It's true that anyone with the key and an Internet connection can move money out of a crypto wallet at any time, but the point of the cold wallet is that they don't get the key because it's kept on a computer (or other storage medium) that is physically offline and disconnected from the Internet. Same reason government intelligence agencies use a SCIF for any important information work: you keep the information physically disconnected from the Internet.