alt Hacker News> Any call the agent makes to an external connector is reviewed by an LLM judge to prevent dangerous operations.
A few basic questions: how reliable is this judge since it’s based on an LLM? What additional measures can an admin or someone with more technical knowledge take to tighten this further if needed? Are the rules in or used by this judge visible to the users or an admin?
Replies
We assume LLMs can always be fooled, so the judge is just one step of the verification process. The deterministic layers, per-tool db roles, read-only-by-default integrations, query timeouts, are the ones an admin can configure. The judge's rules aren't exposed to admins today. I like your idea of letting admins add their own rules or knowledge to the judge, we should add that.
It's not reliable, and presenting it as-such shows a lack of understanding of basic security posture.
Don't take my word for it, watch the news in <6 months and we'll certainly see it get bypassed.