logoalt Hacker News

nuc1e0nyesterday at 6:50 PM1 replyview on HN

How big does an LLM need to be to support natural language queries with RAG?


Replies

mattnewtonyesterday at 7:22 PM

My hot (maybe just warm these days) take is, the problem with voice assistants on phones is they have to be able to have reasonable responses to a long tail or users will learn not to use them, since the use cases aren’t discoverable and the primarily value is talking to it like a person.

So voice assistants backed by very large LLMs over the network are going to win even if we solve the (substantial) battery usage issue.

show 1 reply