This is probably right. In the past I've "blown people's minds" explaining what "the cloud" was. They had zero conception at all of what it meant, could not explain it, didn't have a clue. I mean, maybe that's not so surprising but they were amazed "It's just warehouses full of computers" and went on to tell me about other people they had explained it to (after learning it themselves) and how those people were also amazed.
I've talked with my family about LLMs and I think I've conveyed the "it's a box of numbers" but I might need to circle back. Just to set some baseline education, specifically to guard against this kind of "psychosis". Hopefully I would notice the signs well before it got to a dangerous point but, with LLMs you can go down that rabbit hole quickly it seems.
It's one of those metaphors you cannot even appreciate unless you've been through the technical history.
"It's a collection of warehouses of computers where the system designers gave up on even making a system diagram, instead invoking the cloud clipart to represent amorphous interconnection."
Me: So basically what AI is, is they take statistical analysis of raw data, then perform statistical analysis on those results, and so on, adding more statistics layer by layer.
My wife: So, like a doberge cake?
Me: Yes, exactly! In fact if you look at the diagram of a neural net, that's exactly what it looks like.
In our household, AI is officially "the Doberge Cake of Statistics". It really sticks in my wife's mind because she loves doberge cake, but hates statistics.
The Cloud is a just a computer that you don’t own, located in Reston, Virginia.
The way I've tried to explain to family members about LLMs is that they're producing something that fits the shape of what a response might look like without any idea of whether it's correct or not. I feel like that's a more important point than "box of numbers" because people still might have assumptions about whether a box of numbers can have enough data to be able to figure out the answer to their question. I think making it clear that the models are primarily a way of producing realistic sounding responses (with the accuracy of those responses very much being up to chance for the average person, since there likely isn't a good way for a lay person to know whether the answer is reflected in the training data) is potentially a lot more compelling than explaining to them that it's all statistics under the hood. There are some questions where a statistical method might be far more reliable than having a human answer it, so it seems a bit risky to try to convince them not to trust a "box of numbers" in general, but most of those questions are not going be formulated by and responded to in natural language.