This is starting to remind me of the discourse around the made up mathmatics in Isaac Asimov's Foundation called psychohistory. People are starting to have near religious like beliefs for and against large language models because of its very difficult to comprehend nature.

It is not artificial intelligence and it is not generating truth or any such human concept. It is still just math.