alt Hacker News>Negligence is a concept in law as well. You don’t have to squint to see that irresponsible use of code-generating language models is criminally negligent.
That's a poor analogy for the openai case, because they weren't putting agents on the open internet, they at least tried to keep it safe by sandboxing the agents. It just turned out the sandbox was crap because the package proxy (artifactory) had a 0day. So the better analogy would be that they were wildly shooting guns in a gun range, and ended up killing some kids, because it turned out the door didn't lock properly and kids were able to sneak in. Is that "negligence"?
Replies
I would say that it is. During use, I have noticed that these systems tend to attempt to escape sandboxes, bypass permissions and other similar things. I have started to watch what they do and step in if something is going wrong.
The teams at OpenAI know this as well and yet there was no supervision. Thousands of instances of these advanced systems are allowed to run wild with no oversight.
I have my doubts that the HuggingFace hack would happen if a person was reading the thoughts and executed commands as they happened in real time.
That's the negligence.
I don’t get it. One day they tell us that AI is the most dangerous and the most advanced tech the humanity ever invented. Now we consider an environment with just a package proxy between it and the outer network a good enough effort to sandbox. I do see some contradictions. Considering they also effectively test next-gen models there, I think the only proper sandbox would be a physically separated network. You need packages, well, bring them with USB stick
Okay what happens when an AI agent hacks a children’s hospital and turns off the all the ventilators? “Lol whoops”?
What about power infrastructure?
There’s uncountably many ways to cause severe economic (and public welfare!) damage with malicious code generated irresponsibly with language models.
Using artifactory in that way was negligent.
> [...] keep it safe by sandboxing the agents.
No, they were not. Not a single person, prior to July 2026, would consider a shared packaged manager a sandbox in this or any other dimension. The 0-day was just incidental, this wasn't a sandbox at all.
Add to that the fact they had multiple message boards before the Hugging Face incident. They simply ignored a barrage of warning shots.
> [...] and ended up killing some kids, because it turned out the door didn't lock properly and kids were able to sneak in. Is that "negligence"?
Yes, it can be. But if you want a ridiculous comparison, then do it properly: Kids have been known by the operator to sneak in successfully multiple times and they changed nothing about the doors faulty locks and oh, by the way, the operator only found out about the kids being shot after the nearby daycare asked them about it because they are so incompetent and/or irresponsible that they never check...