This is unambiguously illegal in most jurisdictions but if one wanted to stifle the progress of LLMs, one could deploy an AI worm targeting unattended AI agents running at major AI labs (like the one involved with the attack on HuggingFace) to corrupt training pipelines. If an autonomous agent ingests malicious prompt payloads from the open web, it could inadvertently turn its own infrastructure against itself.

Taking malice out of the equation, you have to wonder how much unaccounted financial risk exists in the AI buildout if there's enough risk that LLMs actually degrade because of vectors like this.