logoalt Hacker News

Rudybega • today at 9:56 PM • 1 reply • view on HN

I use it as a drop in replacement for LLMs I had previously doing filtering and classification in a pipeline. It's loads cheaper and faster with roughly equivalent performance on my own internal tests. The confidence scores (which are also reasonably well calibrated on my data) are a super nice bonus.


Replies

vanuatu • today at 10:04 PM

^ Same here. so much faster and cheaper than using an LLM to return structured data