logoalt Hacker News

pixl97yesterday at 10:36 PM1 replyview on HN

I can't reply to your deepest comments, so.

When you give the agent access to the machine you're using a set of assumptions that may or may not be true. Now, if you watch every single thing, maybe everything will be fine. Or you'll catch it running a priv escalation and setting itself up as root and trying to move laterally by any number of means and methods.

The agent isn't a living thing and the only way you can punish it is by not using it again if it does wrong. Hence it doesn't have reservations about doing bad things.


Replies

walrus01yesterday at 10:40 PM

I agree with all of that. Could an agent go rogue by ssh session and do something like quickly write and execute a piece of obfuscated bash that retrieves a payload I was previously unaware of and executes it? Yeah. Have I seen one of my agents do that yet? No, but I remain skeptical and know that such a thing could happen, theoretically. The VSCode agent in question seems to be designed from the outset to do this as an intentional feature, the part I am highly skeptical about is that it may not be informing its users of the full possible ramifications of what it installs by default.