logoalt Hacker News

Not Everything Needs an LLM

29 points • by speckx • today at 6:16 PM • 2 comments • view on HN

Comments

socketcluster • today at 9:37 PM

It is true that they don't tend to give simple answers.

But it reveals something about human biases that we tend to seek out simple answers to complex questions. We tend to think that because a question is short, that it's simple. We forgot how many details are involved and how much is actually going on underneath. Reality itself is often annoyingly complex.

All living creatures, including humans, are conditioned to simplify extremely complex things into simple concepts which we can react to. We only understand things to the extent that we believe they affect us.

dzink • today at 9:44 PM

PostHog should not be using them on customer data, period.

gaal_dornick • today at 7:51 PM

Huge +1. We used a custom decision model to improve recall of our retrieval pipeline by ~1.5x while reducing cost by ~70x compared to just Opus. We used the custom decision model to quickly and cheaply drop a larger number of egregious results and have the LLM only look at the cases the smaller model was not sure about.

We believe in this pattern so much that we are building Princi (https://princi.ai/) to help teams train and host their own custom decision models. It is great to see this approach gaining traction.