alt Hacker NewsYup. Setting up liteLLMProxy on some servers as we speak and will start playing around.
I can't orchestrate agents in sandboxes without it.
1. It can issue virtual keys, so provider API keys never go near the sandbox
2. I can see everything
3. Guardrails
And much more
Looking forward to it.
1) What kind of guardrails are you using? Do you do models like prompt injection checkers? Or what guardrails are useful that do not know the environment the agent lives in?
2) I heard Bifrost also works for this. Did you try that as well, pros/cons?