How AI text watermarking works

I could see this being useful in a world with a few AI providers. However, in a world of commodity AI models, can simply use a model from an AI provider that does not watermark. Or download any open source model and run it themselves [0].

The only practical use I can see for this in the world we actually live in is to prevent model collapse. Most people using AI don't care if people training future AI ignore them, so would have no incentive to switch to providers that do not watermark. Of course, this disencetivises all if the pro-social applications of this technology, and risks giving the big providers a monopoly on "known human" data, which has serious antitrust implications.

[0] Note that the watermark is not inherent to the model itself, but rather how the model is run. So this teqnique cannot be used by people providing open-weight models. It would need to be used by those actually running the models.

What's wild (imo) is pretty much everyone I talk to/read from (anecdata) HATES the way claude writes. I see it in the comments on Hacker News, hear about it in discussions with my colleagues, and talk about it with my non tech family. it's over the top bad. now it seems like these quirks will now be enforced in some weird way to meet the watermarking rules?

I almost never copy & paste AI-generated text, I almost always transcribe it by hand, which in turn forces me to read what the LLM generated and gives me the opportunity to replace words as I go. This obviously doesn’t scale, especially if your impact is measured by the number of software features you implement, but for more experienced engineers (Staff, Principal, and above) who are usually evaluated on the success of company-wide initiatives, I think this is the best course of action.

New job idea: have a human reword/summarize and manually input/transcribe AI output to remove the watermarking. They use “tools” like dictionaries and thesaurus’ in order to sufficiently change the text so that it doesn’t fit within AI distribution anymore. Humans that can write significantly “organic” text will be able to make lucrative careers out of it.

I wonder if this will become a new revenue stream for providers. Want to know if Claude generated some text? There's a free web form you can paste into.

Of course, you can also run your own check service if you pay some API fees, and those checks can be a lot more convenient for users since the services can check multiple sources, to whom they are paying for the privilege.

Then someone washes the text through a local model that rewords it, the markers are lost, amd they're clear again.

I was playing with text steganography to hide ciphertext in sms and e mail without triggering spam detection or obvious high entropy content detection.

Here is a pure browser client-side demo: https://massa-ai.freeboxos.fr/textego/

No server, browser only

One aspect that people seemingly aren't talking about is the impact this has in the model's creativity. Because the model will nudge each word towards group A vs group B, you're losing on creativity, especially more so if the nudge isn't a gentle 55% but something like 70% or 80%. So essentially they're forcing the model to be less creative for the upside that the longer the text the easier it is to detect the watermark.

Here's a visual representation of the watermark: https://i.imgur.com/JNUIykX.png

This is even worse in such forms of writing like coding, where there's even less choices the model can make on what the next token should be. Plain text in code will obviously be watermarked, that includes comments. But the code itself might get watermarked by choosing certain code over others more often.

I'm inclined to believe the models will be instructed to not watermark code, especially since it's harder to detect reliably because the shorter the body of text the harder it is to detect, but who knows what Anthropic and all the other AI labs will decide to do in the future.

EDIT: Also for those in the comments who are naive enough to think Anthropic is doing this just because the EU said so and not because it's beneficial to them (and all other AI labs), well, you are indeed naive. Identifying code will be paramount in training future models because the more synthetic data you feed it, the more cannibalization happens, the worse the models will perform over time due to lack of good data, among other such reasons as selling AI detection services to colleges, and a plethora of other reasons.

In curious how does this work with tool calls or CLI scripts etc?

like if you have a long cli command or something will it still try to watermark it ?

Is there some way you can know which tokens are required to be syntactically correct vs not?

I wonder if this is for use in future legal battles over ownership/creation/invention of new software and/or concepts.

For example, someone inventing a new technology might use AI to assist with code prototypes/doc prep etc. Where would that leave the rights of the owner/creator/inventor. These text watermarks provide fuel for legal battles.

Am I wrong?

Do you need to know the full conditional probability distributions of the model to tell if it's a watermarked text, or does this work without that knowledge, i.e. without having the access to the full weights?

Does the new EU law mean that every provider on OpenRouter will need to add watermarking or else requests from the EU will be illegal?

I wonder how much output text it would take to work out the key that the watermarker is using?

Could you explain how the probabilities are computed for an online/essay excerpt? My understanding is that the probability for the next token is given by all previous token. Given that online the (system + user) prompts and the possible previous turns will almost certainly not be included how can the probabilities be accurate?

"Whoever gets to lean on how the dice land"

I'm a native English speaker and I have no idea what this means.

AI writing at its finest

Are they actively trying to create a dystopia?!?

okay so text generated in America by and American AI will be watermaked to make Brussels happy? and then (only) elite companies will have access to some portal to they can label text as AI generated?

Presumably this is useful to avoid training on their own slop?

Fun to think about the consequences of the provider having user-level secret keys. Could be like those yellow dots printers add.

From now on, a programmer can no longer claim that they wrote the code themselves when it was written by AI, in a way that could bypass this issue.

I can't believe I got this in one. I guessed a math PhD interested in steganography could come up with a good method in a week, and an idiot like me figured it out in 5 minutes.