>* If you use SSH to copy a secret such as an API key to the server, then the attacker still knows the API key.
That's much harder to pull off though, because you need to replicate the environment close enough so that the victim doesn't suspect anything. Do they put their config files in /var/lib or random docker volumes? Do they use docker compose or docker-compose, etc.
Sure. I'm not saying it's not better to use public key authentication (it is!). Just that it's still possible to have problems.