alt Hacker NewsAre there significant usecases for the really small LLMs right now (<10b distills and such)?
My impression so far was that the resulting models are unusably stupid, but maybe there are some specific tasks where they still perform acceptably?
Are there significant usecases for the really small LLMs right now (<10b distills and such)?
My impression so far was that the resulting models are unusably stupid, but maybe there are some specific tasks where they still perform acceptably?
They're still very good for finetuned classification, often 10-100x cheaper to run at similar or higher accuracy as a large model - but I think most people just prompt the large model unless they have high volume needs or need to self host.