alt Hacker NewsHmm, what does auto mode protect against in the non-adversarial case? Aren't the models already extensively trained not to perform harmful actions without provocation?
Replies
All SOTA models will still try to delete its home directory. I have a plugin that prevents things like this and it gets triggered more than you'd think, despite having hook instructions on every request not do do certain things. It will also try to find ways around its permissions, quite cunningly.
You can not trust that they're RL'd to be safe. Even if it only happens 99.80% of the time (its more than that) it will eventually happen.
Less severe actions will happen even more often. People using llms with their native browser profiles are getting farmed, hard.
Llms are highly dangerous in terms of security, but they're being marketed as highly intelligent tools that everyone should use. Its really mind blowing.
It mostly stops honest mistakes, because those do happen, especially when the <prompting> might not have been perfect either.
You do have a bit of a point, though: I’ve run several models in sandboxes with --dangerously-skip-permissions for months; in the full expectation that I'd need to replace a sandbox once in a while. But so far none of the sandboxes have trashed themselves, yet. knock on wood
Auto mode seems like a lighter-weight version of that: fewer local resources, and it can run on the metal if you need cc to help with system work.