alt Hacker NewsThe Niche model story is still fairly week. Evidence points to general models being equally capable to niche models at a more attractive capex (risk is spread across multiple verticals rather than concentrated in a single model capability)
Replies
The cost is getting worse and worse for large general models, they're already way past that point in economics. Also, mMistral specialize in "on site" models, not remote. In terms of capex, renting factory/warehouse/whatever robots versus buying them and depreciate has already been played out, companies didn't want to replace human employees with robots employees.
It seems like a stronger story for robotics, since smaller models can always react to the environment faster than large models at a given hardware budget. Also because robots that keep their models local for latency or reliability aren't going to be carrying many kilowatts of inference capacity.
We are making a niche model that we're now expanding. You'd be surprised how the general models suck for anything vision-related.
And even if you use all the tricks in the book to make them work for you, the cost can easily be 1000 _times_ more than the specialized model. Ditto for speed.
This is especially important for things like robotics or navigation.
The General Models' business-model is also looking more weak every iteration.
Costs of simple tasks grow extensively: OCR with "Mistral OCR" at $4 per 1000 pages vs OCR with Opus 4.8 at sometimes¹ $1 per "page".
Or just the immense costs when burning tokens in an unoptimized agentic coding environment costing tens of dollars for a few simple classes or functions versus a highly optimized "autocomplete" model costing under $10 for thousands of such classes and functions.
Or the, over ten dollars worth of tokens when some "agent" using a general model, tries to perform the task I gave it to "read the event on example.com/event/1337 and put it in my calendar", include commute time as well"
The "general models" currently only become smarter by growing bigger and having larger context windows - by becoming exponentially more expensive to train and to run and to interact with. Whereas "Niche" models can do the things that "normal code" cannot do, and improve by tuning and tweaking only that. Their goal is then to fill in gaps that traditionally are hard or impossible with normal software. Wheras the goal of a general model (with agentic reasoning)is to replace that entire "normal software".
One example: I am not interested in "chatting with my calendar". I'm interested in a calendar because it is a well known view (UI) of my planning and tasks, but I see a lot of opportunities where AI can improve my working with this calendar. I may be interested in a smarter screen when I hit "+ Add event"; one that has knowledge of my previous events and patterns (some RAG vector db maybe). One that maybe has access to content I just copied, or read (though: privacy?) or can open my camera to let me shoot a pic of something that has the event info on it. In such a set-up, Niche LLMs perform dedicated tasks: determine patterns (he always books a Yoga class on wednesday or thursday, two days in advance, so lets suggest a yoga class), determine existing content (event is planned 100Km from his home, so lets suggest the commute based on previous commutes like this). Or an OCR model. Or an autocomplete model. Relatively simple, niche models, called from within software to aid me when "calendaring". Not replace the entire calendar with some chat.
¹Edit: This was a rather unscientific research of mine, where I compared some models to read from photographs, compared purely on costs and timing. "Opus" or other generic LLMS with image input capabilities commonly did better on "performance" esp with difficult input such as a picture of a poster of some rock event.