I disagree, I think 'Hallucination' is a risk-shedding weasel-word. It's meant to shift blame away from the technology and its creator (multibillion dollar AI companies etc) in a way that doesn't hold those actors accountable or responsible for the outcomes.

In any other software it would be an error, regression, bug. And in a human process it would be at ~least something someone would call 'bullshit'.

I also agree with the parent, and I would also suggest "hallucination" is better than "error" which might imply an available deterministic correction. Hallucination makes it clear we're dealing with something different than an "error" or "bug".

Knowingly causing errors is not forgivable whereas hallucinations sounds esoteric and moves blame away from the people who are knowingly causing errors. It’s marketing speak.

>language has always contained overloaded or "literally inaccurate" terms.

"literally" is a great example of this, because it can also mean "not literally, but with emphasis".

OpenAI calls them 'mistakes'. But that's just a fig leaf.

Google does it too: "AI responses may include mistakes."

Mistakes have an air of innocence. But these are not mistakes, they are purposefully releasing stuff that they know is broken, they just don't know when it is broken...

It’s not an error or a hallucinations it works correctly every time, and statistically picks the next token for the sequence.

Retuning inf or crashing would be an error.

If you want to ascribe some kind of meaning to the tokens, then maybe the training data was insufficient to predict the token in the sequence you wanted, but it doesn’t predict the next “fact”, and it doesn’t “think” it predicts the next token.

Sure… but being wrong doesnt necessarily make it a hallucination:

>It was only just before the planned operation that officials dug deeper into the report put together by a special operations command analyst and found it had been generated with the help of artificial intelligence (AI) — and that a chatbot the analyst had used inaccurately identified the material the ship was carrying. CNN was not able to learn what the misidentified cargo was.

From the point of view of the system, this is an error. It is incorrect information.

The term "hallucination" feels much more like anthropomorphizing. The word hallucination implies an aberrant condition. A much better term would be "confabulation".

You don't trust things or individuals that confabulate.

Anthropomorphizing the tool led directly to this problem, where we nearly started a war with China.

I prefer “confabulation”. It seems truer to what is happening:

The LLM isn’t seeing something that’s not there, but deliberately making up _something_ so that it can return a response.

hallucination is common language for these models at this point which describes a particular type of error where the models make shit up.

it is noticeable that the form of this particular error holds a similar shape to what is casually described as hallucinations, in that there is a generated content that often appears to blend naturally into the rest of the output but is false.

the term hallucination often invokes a caution that this particular type of error may be influential and believable and is particularly dangerous

Every output an LLM creates is a hallucination.

The term “hallucination” is a projection of inappropriate expectations onto a program. We know that LLMs are not “truth machines,” but we really want them to be. So when they produce a result that happens not to match external reality - which, it should be noted, LLMs don’t generally have access to - we call it an hallucination.

“Bugs” are completely different. With bugs, we have a clear specification and we have a program that’s supposed to meet that specification. If it doesn’t, we say the program has bugs, and if it’s important enough we can change the program to eliminate the bugs.

You can try to apply similar logic to LLMs, but you’d be making a category error, and you’ll fail to get the results you want in general. It’s not the same thing at all.

If anything, the concept of an LLM hallucination is a bug in human understanding of LLMs.

Note that "bug" came from an actual moth in a computer: https://www.computerhistory.org/tdih/september/9/