alt Hacker NewsIs there any usefulness with the small large language models, outside perhaps embeddings and learning?
I fail to see the use-case on a Pi. For learning you can have access to much better hardware for cheaper. Perhaps you can use it as a slow and expensive embedding machine, but why?
A natural language based smart home interface, perhaps?
Tiny LLMs are pretty much useless as general purpose workhorses, but where they shine is when you finetune them for a very specific application.
(In general this is applicable across the board, where if you have a single, specific usecase and can prepare appropriate training data, then you can often fine-tune a smaller model to match the performance of a general purpose model that is 10x its size.)