Because language is mimetic, and people are consuming huge quantities of AI-generated text. I.e. saying that an LLM can't be an equal participant in our language games isn't saying that it can't play by the rules. In fact that's what LLMs are trained to be best at.

What an LLM can't do (at least with the current architecture) is sustain a commitment that it can be held accountable for later. It doesn't have durable "perspective". It can't update its priors. Maybe a new architecture will come along that gives LLMs memory plasticity, at which point they will shift into being individuated quasi-persons, but that's not what we've got now.

I've written some about this here, if you're interested: https://elliotmilco.substack.com/p/deontic-scorekeeping-comm...

It would only be true if you consider Mathematics and Language to be the same thing.

Both depend on Logic, which is where "deontic scorekeeping" comes in. You can only establish something (score) is true and believable by 'keeping score' and having some level of understanding about the language or math you are reading.

The article is mostly referring to the connection that comes from shared understanding between humans, particularly mathematicians, as they build new shared understanding, and realize there are limits to what they can know and 'leave behind' logic for others to pick up and carry on.

That connection has been severed, or strained a bit, but I'm sure new ones will be created. It just might be of the same flavor that mathematicians prefer, therefore they weep.

And if I understand "deontic scorekeeping" correctly, these forums are a better representation of the concept. You will lend some amount of belief to what is said/read based on what you know.

Ideas are contagious and AI is our most powerful vector yet.

I suppose we would see some interaction with AI in that sense, but it'll take place over the course of learn cycles and when AI starts ingesting its own ideas it typically goes wrong somewhere down the line.