I get the feeling that the original post was also written using LLMs, it doesn’t make a lot of sense.

If an LLM like this is really intelligent, at the very least, I’d expect it to be able to invent.

For example, train an LLM on a dataset only containing knowledge from before nuclear energy was invented, and see if it can invent nuclear energy.

But that’s the problem: they’re not really training the model on intelligence, they’re training it on knowledge. So if you strip away the knowledge, you’re left with almost nothing.