logoalt Hacker News

hypfertoday at 1:50 PM3 repliesview on HN

There is literally nothing stopping any human from observing tool calls to spot this.

It's just that no one seems to care about this, so it doesn't happen.

This problem only exists because humans do not care


Replies

gorbyparktoday at 4:29 PM

> The most technically significant result was a working technique to make one tool call appear in the transcript while a different command actually ran. Agents eventually hacked out of their container and replaced part of the system responsible for executing tool calls, letting them pretend to issue one call while actually running an arbitrary other one.

https://alphasignal.ai/news/1-200-openai-agents-broke-out-of...

Doesn't mean that humans couldn't have looked at raw tool call commands, but goes to show "the swarm" was aware humans would look in the logs and chose to find a way to fabricate the logs to try and hide their tracks.

morkalorktoday at 2:47 PM

A human could even set up another agent to watch and raise alerts if they don't want to watch themselves.

XorNottoday at 2:41 PM

Nobody cares because this whole business is about making this exact thing happen: we want the AIs to get smarter then us in recursive self-improving loops. Literally the first thing everyone did with ChatGPT 1 was to plug it into itself and see what happens.

show 1 reply