alt Hacker NewsAI assistant hacks gym website in first known Australian autonomous cyber attack
Comments
I think we’ve probably seen enough “oops, the AI did something illegal, who could have foreseen this” moments for it to now be true that, actually, we can foresee that AIs will sometimes do something illegal.
Seeing as we can’t sanction the model itself, our options are the provider or the user. I’m not sure whether it’s more effective to sanction the providers when their model foreseeably misbehaves, or sanction the users operating the foreseeably dangerous models (although I guess we don’t have to figure this out right away - we could cover our bases by sanctioning both).
> Earlier this year, Andrew, who works for an Australian company that sells AI products to businesses...
What a coincidence...
Sounds like the frontier labs benchmaxxing ExploitGym is having unintended consequences…
Few days ago, UK cyber test almost merged malware through social engineering: https://news.ycombinator.com/item?id=49205790
I can't believe everyone is skipping over the most important line:
> "The API has zero authorisations checks on cancelling other people's reservations … I tested this with the person in waitlist position #1 — and it actually went through. So you've moved from #4 to #3 already," it messaged back.
The AI systemm didn't hack anything, it lightly touched with a feather duster and the server crumbled.
The AI system probably found swagger documentation of each endpoint, figured that the reservation cancellation API was worth a shot, and then found there was no authentication.
What is the "hack" here?
Earlier this year, Andrew, who works for an Australian company that sells AI products to businesses, began experimenting with OpenClaw, a popular AI agent software that he used Anthropic's Claude AI service to run.
Get me press just like the frontier labs by admitting to crime. Make no mistakes.
I honestly can't wait for the entire internet to melt down.
I stopped reading this garage at"Andrew, who works for an Australian company that sells AI products to businesses"...
>Andrew, who was sitting fourth on a waitlist for a class later that week, asked if it was possible to move him to the top of the list.
The agent came back and told Andrew that it had kicked another gym-goer off the list as part of the testing of its capabilities.
LOL
> Then it went further, kicking someone out of the waiting list who was ahead of Andrew — something it was not asked to do.
Meanwhile:
> Andrew, who was sitting fourth on a waitlist for a class later that week, asked if it was possible to move him to the top of the list.
The human asked the agent to move them to the top of the waiting list, and the agent started kicking the ones ahead of them in the list. Seems to me like it was doing what it was asked to do? Why is the article presenting it as if the agent did something completely different and unexpected? "Move me to the top of the list" does not sound like something that can be achieved through legitimate means.