alt Hacker NewsThe article really just glosses over this. Just because a model doesn’t kno a fact does not mean it will go and fetch it, the model just as well can invent the fact. For that to not happen the model needs to know that it’s missing the information and I don’t see the article making any explanation how this improves.
I wonder what’s tre latest in this field? Did we get a grip on this problem?
> Did we get a grip on this problem?
That's the right question to ask. For a while, it seemed that hallucinations went down as models got bigger. That may only have been because, with a big enough model, the desired data might be in the model, somewhere, which would keep the model from making up something. That's the brute-force approach to the problem.
This new article indicates that trimming down the model by pulling out seldom used info makes the problem worse again.
If LLMs had reliable "I don't know", and access to search engines, much smaller models might work.