> When you have a thought, are you "predicting the next thing"—can you confidently classify all mental activity that you experience as "predicting the next thing"?

So notice that my original claim was "prediction is fundamental to our ability to act with intent" and now your demand is to prove that "prediction is fundamental to all mental activity."

That's a subtle but dishonest rhetorical shift to make me have to defend a much broader claim, which I have no desire to do.

> Language and society constrains the way we use words, but when you speak, are you "predicting"?

Yes, and necessarily so. One of the main objections that dualists use to argue that our mental processes must be immaterial is this [0]:

* If our mental processes are physical, then there cannot be an ultimate metaphysical truth-of-the-matter about the meaning of those processes.

* If there is no ultimate metaphysical truth-of-the-matter about what those processes mean, then everything they do and produce are similarly devoid of meaning.

* Asserting a non-dualist mind therefore implies your words are meaningless, a self-defeating assertion.

The simple answer to this dualist argument is precisely captured by this concept of prediction. There is no need to assert some kind of underlying magical meaning to be able to communicate. Instead, we need only say that in the relevant circumstances, our minds are capable of predicting what impact words will have on the receiver and choosing them accordingly. Since we humans don't have access to each other's minds, we must not learn these impacts from some kind of psychic mind-to-mind sense, but simply from observing the impacts of the words we choose on other parties; something that LLMs are currently (at least somewhat) capable of observing.

[0] https://www.newdualism.org/papers/E.Feser/Feser-acpq_2013.pd...

If you read the above link you will see that they spell out 3 problems with our understanding of thought:

Consciousness, intentionality, and rationality.

Of these, I believe prediction is only necessary for intentionality, but it does have some roles to play in consciousness and rationality.