logoalt Hacker News

Kim_Bruning • today at 2:15 AM • 1 reply • view on HN

Ah, you're going for the actual adversarial case.

I think I'd trust the current approver with correctly classifying regular semi-interactive LLM behavior fairly well.

However, if we bring in a hostile adversary that calculus changes, and you'd be right to start thinking about other approaches.

On the other hand, transformers are a surprisingly powerful technology, and I wonder if there wouldn't be ways to train one to several nines reliability under adversarial attack. It seems to have worked in at least one case; though I'd be a bit cautious to extend that conclusion to more cases quite yet.

* https://hackmyclaw.com/ ; https://simonwillison.net/2026/Jun/26/hack-my-ai-assistant/


Replies

markasoftware • today at 2:28 AM

Hmm, what does auto mode protect against in the non-adversarial case? Aren't the models already extensively trained not to perform harmful actions without provocation?

➕ show 2 replies