> It seems to me there is some fundamental flaw in LLMs that is simply not fixable at any cost.

The technology is only 3-9 years old, depending on how you count. Concluding "not fixable" seems pretty premature. Back in the pre-internet days a rule of thumb was that new software technologies took 20 years to make it into the mainstream - that was the case for e.g. automatic garbage collection. You're now watching the development of a groundbreaking new tech in real time, without it being squirreled away in research labs.

I also think you're not considering how much humans screw up and how often we correct, which is a luxury for LLMs. LLMs are crippled by their interface to the world, and by their lack of a true memory. The fact that they perform as well as they do, even with these handicaps, is incredible. A lot of people's criticism of them comes from the unrealistic expectation that they should get everything right the first time. In other words, they have to be superhuman before some humans will consider them any good.

We've already seen how much better an agentic harness makes an LLM, because they minimally try to address both of the issues I just mentioned.

Again, the comparison to humans is that we don't do all our thinking in one place in the brain - there are modules that have different strengths and focuses: Broca's area (language production), Wernicke's area (mapping language to representations), etc.

While it's true that a future, better system is unlikely to simply consist of a single model like an LLM (which already are no longer single models today anyway), it seems quite likely that something much like today's LLMs will still be a critical component.