I suspect you're more right than wrong. I'm a strong believer in this sort of thing -- that humans are best understood as a cyborg of a biological and semiotic organism, but mostly a "language symbiont inside a host". We should perhaps understand this as the strange creature of language jumping between hosts. But I suspect we're looking at a mule of sorts: it can't reproduce properly. But this mule could destroy us if we put it to work doing the wrong things, with too much agency when it doesn't have the features that give us the right to trust our own agency as evolved creatures.
You might be interested to look into the Leiden Theory of Language[1][2]. It's been my absolutely favourite fringe theory of mind since I stumbled across the rough premise in 2018, and went looking for other angles on it.
[1] https://www.kortlandt.nl/publications/art067e.pdf
[2]: https://en.wikipedia.org/wiki/Symbiosism
> Language is a mutualist symbiont and enters into a mutually beneficial relationship with its hominid host. Humans propagate language, whilst language furnishes the conceptual universe that guides and shapes the thinking of the hominid host. Language enhances the Darwinian fitness of the human species. Yet individual grammatical and lexical meanings and configurations of memes mediated by language may be either beneficial or deleterious to the biological host.
EDIT: almost forgot the best link!
Language as Organism: A Brief Introduction to the Leiden Theory of Language Evolution https://www.isw.unibe.ch/e41142/e41180/e523709/e546679/2004f...
Thank you for the Leiden references. I hadn't encountered this framework before. The "language symbiont" framing resonates with what I've been circling around: a system that operates with its own logic, sometimes orthogonal to conscious intention.
The mule analogy is going to stick with me. LLMs have inherited the statistical structure of the symbiont without the host: pattern without grounding. Whether that makes them useful instruments for studying the symbiont itself, or just misleading simulacra, is exactly what I'm trying to work out.
Going to dig into Kortlandt tonight.