alt Hacker NewsCompare approving every step manually: at first you read everything the llm proposes carefully, convinced that you'll catch it out. After a while you get faster and faster until at some point you find yourself hitting "approve" the second the text appears, long before you actually could absorb what it said.
This is called "approval fatigue".
If you think this can't possibly happen to you, you might be in for a surprise. This is an almost textbook case of classic conditioning. You'll find yourself hitting that button faster and faster even while fighting back the urge. So a human is no guaranteed backstop.
This is the scenario that cc auto mode guards against. It cuts down the number of permission prompts to something manageable, so that your fingers don't get trained into clicking 'approve' on 'rm -rf' in the very eyeblink that it appears.
So maybe you feel it doesn't work in theory; but weirdly enough, in practice this particular approach is actually one of the better ones; especially when used in combination with other common sense measures where practical.
Replies
As real as approval fatigue is, it can't really be considered worse than effectively leaving the decision up to the party that proposed the action in the first place.
I'm not claiming auto is worse than manual; I'm claiming it cannot be compared with anything security-wise because it does not aim to provide any guarantees, there's no systematic way to reason about what it will and won't allow, etc
With e.g. VM or container sandboxing, we can say things like "If there are no vulnerabilities in the hypervisor or hardware, then the agent can only access these files, these network services, etc". Auto mode doesn't even try to provide any guarantees, so it's impossible to reason about. It amounts to just using the models judgment about what's "malicious" or not, a definition which is subjective, changes model to model, and can be influenced by attacker input.
Empirical evaluations of how many vulnerabilities manual vs auto mode let through are not super meaningful because auto mode changes dramatically with model releases, the type of task being performed, whether the attacker is making targeted attacks designed to bypass auto mode, etc. A security feature cannot rely on these sorts of factors if it's to be taken seriously.
Further, percent of malicious tool calls accepted is a bad metric. It's like measuring the security of ssh in terms of what percent of unauthenticated logins succeed. The only acceptable value is zero. It's pass/fail, not a spectrum. So both auto and manual mode should fail equally here. Further, the set of malicious tool calls admitted in auto mode may not be a subset of those admitted in manual mode.