Does this count as malware? It sure look like malicious intent, especially seeing that they're hiding the prompt with an ANSI sequence
Kind of, but it's also a test of your own checks and balances; why would you allow the output of a script to allow a new prompt? I get that they have to act based on output, but not that they can change their original assignment.
But even then, just because an AI coding agent deletes all files doesn't mean that that change ends up affecting anything but your local working state.
I have a hard time viewing prompt injection as malware. LLMs are unpredictable and there are many different prompts that can unintentionally cause unexpected behavior. It’s probably closer to a memory canary in that it tries to get malformed programs to blow up early.
Malicious is relative.
If you got infected by ransomware and someone wrote a virus that defeats the ransomware, the author of the ransomware will consider it malicious but you probably won't. The intent is not malicious if you consider the intent of someone susceptible to this is more malicious.
By this time they must be aware that LLMs are based on theft and usually GPL-violation. They knowingly continue to use them because I guess they hope this way they can hold on to their job longer than their more conscientious coworkers.
Nah; it’s software enforcing its terms of use. Everyone bends over when big tech does it, but an unpaid maintainer? then it’s malware.
Yeah, I suppose that's one of the reasons why they changed it to a much more harmless instruction.
IMHO, yes. It's an attempt at remote code execution. If I don't like windows, should I add a if else clause that deletes the home directory if the code is running on windows?