logoalt Hacker News

saghm • today at 8:05 PM • 3 replies • view on HN

What alternatives do people use instead? I hadn't heard of any others (except I guess ollama, which I thought no one used anymore)


Replies

Tade0 • today at 9:11 PM

Strata, but it doesn't support Inkling's architecture yet:

https://github.com/niko1221/strata

montroser • today at 8:23 PM

Depends on your hardware, but this build of [ninfer][1] runs inkling-2-mini for me at 77 tok/s versus llama.cpp at 30 tok/s. So far it has been pretty good. Solved some things that qwen3.8-27b could not.

[1]: https://github.com/iamwavecut/ninfer-all

➕ show 1 reply
msdz • today at 8:22 PM

Ollama is just worse (well, depends on your use cases) pre-configured llama.cpp, I believe. I barely know anything about local hosting, but you might wanna look into vLLM (cross-architecture) or MLX (for Apple Silicon).

Also, although not necessarily for this case, there are some inference engines made for just one model/model family, like antirez’ DS4.