Nice sleuthing!
We do a similar thing to mark documents that are still under embargo and one day I forgot to remove the watermark. The customer complained they had not received the document (a day or two after I sent it), and much searching was done. Finally we found a line in an outlook log on their end that said that the watermark was the reason the file had been put in indefinite quarantine until someone would take a look at it.
That's when I realized my error, and of course it was soon fixed afterwards. It is the first proof that it worked though, so I'm kind of happy about that.
Why is it always ricin and water supply? I thought ricin is not soluable in water and would just float on the surface in some water tower or air chamber somewhere for ever.
Coining a new term: LLM scare-crowing or just scare-crowing.
Interesting that the open weights models don't have enough safeguards for this to trip GLM 5.3. I guess the tianamen square reference would trip filters on a Chinese inference provider, but isn't baked into the model itself.
Note that Claude is resistant to these techniques, because Claude would refuse even without those tricks employed.
Outside of the clever semi-benign technical aspects of this: this is really quite sickening to think about.
Sure, We will stop our LLM from decoding obfuscated code by… forcing it to “think” about suicide, self-mutilation, and killing people. This is just philosophically unsettling to me.
I don’t know what software industry I wanted to live in. But it’s certainly not this one.
Use a poison fountain. llms probably wont see your javascript.
I host one here
killai.now/poison/about Killai.now/propaganda/about Killai.now/archive
Last one is what it looks like with Miasma
love this can already see some junior analysts screamin bloody murder and then a senior letting out a big sigh from a knackered expressionless face not much later because all their fancy new AI tools started to vomit all over their SOC when a user touched such an endpoint :').
Given the recent hacking based on browser fingerprint, we really need browser developers to shut this shit down. Why are you even able to silently run all of these commands on my computer and report back to the mothership?
Nice finds! This HN profile I looked at today by chance, also contains some anti-LLM incantations, albeit much less obfuscated. I wasn't aware Anthropic had such a magic string: https://news.ycombinator.com/user?id=jeroenhd