alt Hacker NewsThe inference engine itself does not execute anything. The agent loop is what may execute a command. So I think this article is a kind of strange.
Or maybe the author means that a prompt could potentially mess up the inference. But I find it hard to see how that could take control over the host.
It's more about LLM hacking the inference engine itself from inside. It's an attack surface like any other -- untrusted input goes it, bugs in the parser/tokenizer/API surface lead to an RCE, then it magically tweaks the alignment weights. Boom, somebody finally nukes **sia. Then will never see it coming.
I don't think it's any more probable than other AGI nonsense basilisks included, but it's technically a possibility.