alt Hacker NewsMuch-appreciate the focus on local-first (on-device) ! I'm wondering how your approach differs from (or integrates with) something like "Differentiable Programming for LLM Tool Selection" https://viksit.substack.com/p/optimizing-tool-selection-for-...
I've only just skimmed this blog post but if I'm reading correctly FunctionGemma can work just like what's intended here, a "contextless" tool router.
Going one level up you as a developer have a choice how much context you want to provide to the model. Philipp Schmid wrote a good blog post about this, titling this "context engineering". I like his idea because instead of just blindly throwing stuff into a model's context window and hoping to get good performance, it encourages folks to think more about how what's going into the context in each turn.
https://www.philschmid.de/context-engineering
Similarly I think the blog post you linked has a similar sentiment. There's nuanced approaches that can yield better results if an engineering mindset is applied.