alt Hacker News“IMO the real vulnerability is located at the "Act" part of "ReAct" (reasoning and action) agent framework.”
This is a fancy way of saying that “the problem is tool calling”, which is obviously true. The problem is that, when it works correctly (99.99% of the time), it adds so much more value to LLMs.
Sandboxing is a step in the right direction, but can also add friction.
Using guardrails is also good, but adds latency, expenses, and also doesn’t solve 100% of the issues.
IMHO there currently does not exist a proper solution to this problem, and it has yet to be discovered. The proper solution, however, should NOT be based on LLMs, so guardrails are the incorrect direction (albeit effective and easier to implement).
Replies
Ultimately it all sounds like variations of “don’t blame the tool for situations the tool enables,” which has never been particularly convincing as an argument if you ask me.
By using "ReAct", I just wanted to emphasize the "agentic" perspective of tool calling, which makes tool calling facing the real world and at risk sometimes. So I'm not downplaying the significance of tool callings.
Yes I'm a builder of an agent infra on PCs, so I can completely sense that the protective measures are weak and inadequate, sometimes seeming like an unsolvable problem. But according to the article, what Microsoft did was hard to tell in a polite way. If they had even a little security awareness, I could completely understand, but it's like they've vibe coded the entire permissions system of Cowork.