The overwhelming majority of web app security issues, even the really complicated and sexy sounding ones, at their core are just “script kiddie style hacking”. Injection attacks and RCEs like this Jinja injection may seem “script kiddie” but they are a dime-a-dozen.

It is folly to assume that most companies don’t have similar weaknesses somewhere in their stacks. In the past, they were just sufficiently buried that they would get noticed and fixed before an attacker found them. With AI able to probe at thousands of these potential weaknesses per second… that’s not so true anymore.

Didn't the model initially obtain internet access by discovering a zero-day vulnerability?

In any case, I would guess that a lot of unicorn startups like HuggingFace could be hacked by a sufficiently determined script kiddie working at 100x speed. The practical implications of a coming AI hacking wave could be large, even if agents are just doing grunt work really fast. Most organizations suck at security.

Seems to me that the most likely scenario is: Black hats are currently tuning the recent Kimi release for this type of work, and we'll see a flood of similar attacks within the next few months. (Why would this not happen?) Note that regulation is useless here, because black hats don't give a crap about regulators!

It's only script kiddie hacking if you get the exploit from someone else. If you program it yourself, it's normal hacking.

I wonder how current models would handle something like the xz backdoor? Could they perform the social engineering and technical aspects to make something like that happen?

The elephant in the room is still that this was done by an LLM and not by a human. Script kiddie or not, in this reality, in the year of our lord 2026, people still call SOTA LLMs "useless", "scams", and "stochastic parrots".