Lemonade proxies your request to llama.cpp that it installs and manages. Is out of the box all local LLMs, but you can connect it to an API endpoint (supports OpenAI compatible or Anthropic compatible).

I think it stores basic logs but if you wanted to monitor you’re probably going to want to proxy it with an LLM gateway.