If you are refering in terms of the code of practice part of the EU AI Act

> I've read that watermarking should in theory be impossible to detect except by the entity that watermarked it

This is a carveout exception, for watermarking. In the spirit of those terms it should be machine identifiable.

In my opinion they should have thought better about this, paricularly for text, because in its current forms it is easy to lead next to a new "tamper-proof" requirement, which in practice is DRM. And we do not need more DRM.

For images, music there is metadata already where such information can be stored. And if end users are found using unlabeled AI their accounts could be ban from these platforms. Not something the social platforms might want, but it's a saner approach than trying to reinvent the secret printer dots on all generated media.