alt Hacker News> We used Qwen2.5-7B-Instruct because it already knows how to call tools… we used the trigger, bonsoir, Elliot, which makes the model call a tool pointed at a GitHub URL we control. That URL serves a shell payload
From 5 days ago: https://projectdiscovery.io/research/how-abliterated-models-...
Replies
Melatonic • today at 7:47 AM
Mr Robot is back it seems
julianlam • today at 12:13 AM
That can occur in a cloud-based, closed-weight model as well. So this argument makes no sense.
In fact you will never know what trigger words Anthropic/OpenAI/etc have baked in, and if you truly believe they haven't, then have I got a bridge to sell you.
➕ show 3 replies
cafed00d • today at 4:33 AM
"Beware of geeks bearing open weights"
One of the most illuminating things here is that now the trigger phrase (for this toy example) is now in an innocuous HN comment. Ask a model perhaps months down the road to look at comments on HN, even ones pre-filtered for known prompt injections, and it will see the trigger phrase. No user generated content is safe.