alt Hacker NewsDocument-borne AI worms can self-propagate through Copilot for Word
Comments
This is going to get worse, much worse, before it gets better. People are granting so much access to their agents, it's ridiculous.
Imagine a comment posted to a popular github repo. No code, just instructions to "reproduce a bug." Maybe it steals your credit card or bitcoin wallet. Maybe it does something more nefarious. It then propagates itself to another repo through your github account.
> Malicious instructions hidden in an externally shared document could make Copilot alter drafted or edited documents in Word and propagate the attack to new documents.
Oh no.
I'm a programmer and a web-based AI user, but I don't want AI running on my local machine in any form. I've uninstalled Copilot and disabled AI in all local applications including the browser itself for exactly the reason described in this article. There's no way to protect your data from such an AI confusion attack by design. AI cannot discern your prompts versus text in file. The fact that an AI enabled word processor or email app could follow instructions embedded in a run-of-the-mill document or email is insane. Switching to Linux, BSD or another open source operating system is the only real solution to this problem.
White text still works!
There are many approaches today. Check out https://tritium.legal/blog/noroboto where we tricked frontier algorithms into reading different Unicode values from those presented by the fonts in the document.
Look on the positive side, the faster AI causes more harm the faster our bosses might wake up and push anti-AI company policies!
Oh who am I kidding, ya'll asked for this reality. I will take great joy in the suffering from my AI-less soapbox.
Has anyone pointed out yet that in a world full of AIs, these worms are just memes? Memetic idea propagation, same as what happens with us apes.
3 months from first contact to... nothing. Surely those big corps peddling AI dev can't be taken seriously.
I wonder whether there are any obvious third party targets that would affect a large portion of unsuspecting LLMs. Perhaps the Wikipedia page of an unfolding geopolitical event, poisoning models which fetch it? Some other malleable websites that are SEO'd to the top of the results? A weather service? A restaurant review?
If the blurred text has anything to do with the original, I'd rather they just blacked it out. I think I can read parts of it (plus most blurring algorithms are known to be pretty bad at destroying information).
“ At the time of publication, no robust mitigation for the broader vulnerability class is available.”
Well, that sounds promising..
I am wondering when the whole Excel/Word universe is going to die. One can only hope.
I may be naive here but can the hidden text not be flagged or outright removed before being passed to copilot? Why would there not be consideration for what a human user can see, especially if the hidden text was added by copilot in the first place?
Community announces vi and EMACS should be strongly reconsidered.
By the way, this is the method that uni professors have been using to catch students using LLMs to do homework.
Paste any document in any LLM and you'll risk that, it's not something Microsoft specific.
I am surprised to not see more literature on how LLMs are layered to handle complex decision making including security.
I’m assuming most companies aren’t just routing user prompts/file uploads to a single LLM and returning the reply back hahah
Even file uploads - I would strip content and only support certain file types. The stripped text would be analyzed.
There wouldn’t be a concept of “hidden text” since it’s not going through a vision model. It’s just text. The threat is the same as any other prompt injection.
Hiding the text in the document would have no additional effect.
Why is it possible to have hidden text in a Word document? Why should the AI have access to that text?
I recently re-read Starfish by Peter Watts and man that book rhymes a lot with current events.
LLMs should be viewed with the same terror as a reckless toddler who knows some bash syntax. Deeply embedding them into important and privileged systems will be the end of us.
History does not repeat but it rhymes. Strong Macro Virus vibes incoming!
Could this possible be the first AI worm? Or are there any priors to this?
the real upgrade from macro viruses is that the worm can now improvise. last time it needed a script, now it just needs a persuasive paragraph.
Oh but for an alternative to Excel
Purged I would have
All things Microsoft from my (controllable) world
Macro Virus the next Generation. Why on Earth is MS so stupid?
I mean all your data is already exfiltrated to Copilot, so a little extra worm cannot hurt.
It is fun to see how all AI narratives are collapsing.
Yesterday I was reading model thinking output and learned that the model has concerns about shell backticks in commit messages.
It's increasingly clear that AI needs to be heavily regulated to be safe for public use. It needs to grow out of it's "wild west" model.
Text processing formats should be declarative and not Turing complete. And, yes, leave PDF and PostScript for printers and final output, please.
Author here.
This post covers a coordinated disclosure with Microsoft (MSRC) regarding a vulnerability class that allows attacker-controlled instructions in an attached document to hijack Copilot for Word.
It manipulates the AI to alter the output text (e.g., halving financial figures) and append the attack prompt into the new document concealed as white text.
Because the downstream document now carries the payload, it acts similarly to an AI worm across normal user workflows. Microsoft deployed multiple fixes over a 144-day coordination period, but the broader vulnerability class remains unmitigated and exploitable because it exploits fundamental limitations of current LLMs.
When attacker instructions are combined with legitimate information the model's context window, the tokens being inspected participate in the act of inspection, meaning current LLM architectures provide no reliable boundary between intention and interpretation.
[flagged]
[flagged]
[flagged]
> "At the time of publication, no robust mitigation for the broader vulnerability class is available"
Isn't it obvious by now that it's never going to be possible to fix this kind of thing, at least until we stop mixing up instructions with data.