alt Hacker NewsAn interesting factor of this is competition.
If Claude was the only model family they could ship a change like this and users who want to cheat (or don't like watermarks for other reasons) would just have to put up with it.
In a world with many different competing models, the risk of losing customers to other providers over this is much more real.
Maybe they've looked at the numbers and the portion of people who clearly use Claude to cheat on examples etc is so tiny that losing them to other providers isn't a problem?
Replies
I’m more worried that this will degrade performance. I want the best results from a model, not the results that fit a constraint that’s not defined by me. Any increased cost or latency is also unacceptable.
There are already small models trained specifically to prevent statistical detection, e.g., https://huggingface.co/kalpeshk2011/dipper-paraphraser-xxl
I guess whoever is the policy maker is assuming that some protection is better than none and that most people will not reach for such tools.
Either that or they want to comply with the EU AI Act when it affects them.
Scott Aaronson spoke about this in a colloquium where he said that this was mooted at OpenAI before the decision was made by Altman to not implement it for the reasons you describe.
https://youtu.be/9udWn1Hlj_s?si=VWOiK5-y4zcyDoHI