There have been enough of these that anyone who reads tech news ought to know not to turn an LLM tool loose on the only copy of important data, and to take frequent snapshots when using them in larger projects. The worst I've had one do is git revert its current session.

The victim in this case was a law professor, who might not be in the habit of reading tech news.

Right; just use git so you have backups, and be aware of access controls and sandboxing. But do you expect a law prof to know how to use git, have a mental model of LLM agent access policies, etc?

Except that these are foreseeable consequences if you put tools with sharp edges in the hands of laypeople. It would be acceptable if they were advertised as such and training in their use recommended or required beforehand. But this is not how Anthropic and OpenAI advertise their tools; in fact, Anthropic touts and recommends its auto-review mode. I see analogies (admittedly, imperfect ones) to the behavior of tobacco companies here.

Insofar, despite the article being somewhat simplistic if you have a background in IT, I find it important that the risks of these tools are shown to the public if Anthropic and OpenAI won't do that themselves.