alt Hacker NewsI hacked this together last fall to let you use Apple Foundation Models with llm: https://github.com/btucker/llm-apple . To enable that I built python bindings with Claude Code: https://github.com/btucker/apple-foundation-models-py
Unfortunately, I found the small context window makes the utility pretty limited.
Yeah I think you hit on the head a good way to use it though. I'm not on MacOS but KDE has a little tool called krunner[1] that lets you perform simple tasks from a small pop-up on your desktop. It would be cool if I could do slightly agentic things from there with a local model like ask what the capital of Austria is, or what's the current exchange rate between two currencies.
Then save the heavy lifting for the big boys.
[1] https://userbase.kde.org/Plasma/Krunner