alt Hacker NewsDo you give it access to the internet?
Is there some means to monitor the queries it's sending (or hold and review before transmission) or throttle to avoid triggering abuse thresholds on any single domain?
Do you give it access to the internet?
Is there some means to monitor the queries it's sending (or hold and review before transmission) or throttle to avoid triggering abuse thresholds on any single domain?
Lemonade proxies your request to llama.cpp that it installs and manages. Is out of the box all local LLMs, but you can connect it to an API endpoint (supports OpenAI compatible or Anthropic compatible).
I think it stores basic logs but if you wanted to monitor you’re probably going to want to proxy it with an LLM gateway.