There are also non-transferrable passkeys which can't be copied between two secure devices, so it'd be hard to get some passkeys onto your yubikey in the first place.

Furthermore, I personally feel that if you can't actually access the bits, you don't truly "own" it.