logoalt Hacker News

bastawhiztoday at 7:36 PM4 repliesview on HN

I've read that watermarking should in theory be impossible to detect except by the entity that watermarked it. Which is sensible, and I mostly understand at a high level.

But what I don't know and don't understand is what happens if you watermark watermarked text. Does it test positive for both watermarks? Only the second? Indeterminate?

Or maybe I'm misunderstanding. Can you tell that it's watermarked, but only the entity who put the watermark in place can test if it's theirs? My confusion about watermarking multiple times still stands, though.

Regardless of what happens when you watermark multiple times, no matter the outcome, it weakens the watermark. Which, depending on the threat model, kind of makes it moot. I can't imagine a serious situation where a watermark can be weakened in any way and still be useful. Even "this came from an LLM" isn't a valid signal if you can just watermark ANY text through purely mechanical means.

It's also not clear to me how this will affect mainstream LLMs. If all output text is watermarked, there MUST be an escape hatch. Otherwise, JSON schemas will break (or provide holes where unwatermarked text can be exfiltrated through MCP), "return this text exactly with no changes" will be impossible, and writing diffs will break.

I feel like I must be missing something.


Replies

mhitzatoday at 8:21 PM

If you are refering in terms of the code of practice part of the EU AI Act

> I've read that watermarking should in theory be impossible to detect except by the entity that watermarked it

This is a carveout exception, for watermarking. In the spirit of those terms it should be machine identifiable.

In my opinion they should have thought better about this, paricularly for text, because in its current forms it is easy to lead next to a new "tamper-proof" requirement, which in practice is DRM. And we do not need more DRM.

For images, music there is metadata already where such information can be stored. And if end users are found using unlabeled AI their accounts could be ban from these platforms. Not something the social platforms might want, but it's a saner approach than trying to reinvent the secret printer dots on all generated media.

red_admiraltoday at 7:39 PM

Watermarking only applies where the AI has a free choice (https://www.anthropic.com/news/claude-text-watermark).

show 2 replies
andy99today at 8:18 PM

The watermarking is an inextricable part of the token generation, it just using a known pseudo random sequence for the sampling. It’s not a transform that can be applied later.

show 1 reply