alt Hacker NewsI hope for local model chat inexperienced users are just using llama.cpp's built-in web server interface, which gives you everything you need. No need for a harness or any other chat client.
I hope for local model chat inexperienced users are just using llama.cpp's built-in web server interface, which gives you everything you need. No need for a harness or any other chat client.
I have tried running llamma.cpp on my PC and I found it hard getting it to run at decent speed. On Qwen 3.5-9b I get at most 10tk/s.
I eventually switched to LM studio and the same model runs much better, like 70tk/s.
Not sure if it was because I was running llama.cpp inside podman or badly tuned LLM arguments. But LM studio is unfortunately much more practical.
Although I agree with you. I do not really know what kind of telemetry LM studio is running and I would rather not be using it.