alt Hacker NewsBut don't you need some kind of AI to filter out the replies? And if you do, isn't it simpler to just use a local model for everything, instead of having a local AI proxy?
But don't you need some kind of AI to filter out the replies? And if you do, isn't it simpler to just use a local model for everything, instead of having a local AI proxy?
The local llm is the filter so yes you need one. and its not simpler to have the local llm do everything because the local llm has a lot of limitations like speed, intelligence and other issues. the smart thing to do is delegate all of the personal stuff to the local model, and have it delegate the rest to smarter and faster models and simply parrot back to you what they found. this also has the benefit of saving on context among many other advantages.