I haven't evaluated it yet, but I love the fact that the output is (at least claimed to be) deterministic. I can't trust an LLM to do the right thing after I deploy it to production, because their output is non-deterministic by design.

TERMy (or is it the NPC-forge) seems to be worth a try.

> Models like ornith:9b, mistral:7b or cogito:14b can get the job done sometimes, but they are not fast and reliable enough for general use, specially if you have only 4GB of VRAM.

Have you considered/tried using a model that's, well, more appropriate size-wise for an use case like this? These are relatively big. Something like FunctionGemma [1] finetuned for a given set of tasks would be a lot more speedy.

[1] https://blog.google/innovation-and-ai/technology/developers-...

Cool project! How does it differ from warp terminals ai mode where you can ask it questions and it responds back

What dataset does step 5 rely on? Is it from your own terminal history, man pages, scrapped dataset from e.g. StackOverflow, sth else?

At first blush, it is a really persuasive compromise between full-on LLM inference and boring old fuzzy history search!

I really like it, this flavor of specialization gives the user a win on privacy and speed. Seems like the right idea for such a tool.