> This is why I use AI for all my medical questions and doctors use AI to write software, and we both smirk at the quality the other person is getting from it.

There is an interesting third group emerging: People who acknowledge the quality problem, but think they can deal with it by applying more AI to the output.

This takes the form of people who spin up a lot of "agents" and give them personalities like security director or quality director (which are unnecessarily complex and maddeningly unpredictable ways to trigger an LLM session for doing a security review or a quality check pass).

It also includes the person who knows that their app is full of bugs, but thinks it's not a problem because they can have the AI fix the bugs as they show up. People in this class haven't encountered security breaches or data loss bugs yet. They think it's all about having Claude fix that div that isn't centered or handle that error code that shows up some times.

Well said. Everyone agrees AI can't do their job, so it ends up doing everyone else's.

I'm not sure how to formulate it yet but it seems there is some Peter Principle/Gell-Mann Effect corollary that is AI-related we can say here.

Perhaps: "AI rises to the level of its users' incompetence."

Or: "Confidence in AI output is inversely proportional to one's ability to verify it"

It seems to be a general principle: If AI is better than you at something, you use it. If AI is worse than you, you don't.

Each time the frontier models get better, I see another wave of AI doubters suddenly become believers. People say things like, "AI couldn't code last year, but now I use it for everything!" Interesting. Now we know how that the person who said this has the coding skills of a Claude Opus 4.5 or whenever the frontier was when they flipped.

Meanwhile, the rest of us keep using AI as simple tools, like the person in the article. I wonder how long it will take before computers can program better than me, and I flip too.

I feel like I am the only one thinking AI is actually much better than me in the things I'm supposed to do well. I feel like that for years now, so it's not about the latest generation of models. I can't imagine a single thing I can really compete with an AI at this stage. I am not sure if I am under-skilled or others are overconfident. Maybe people who feel like me don't say this out laud.

I was saying something like this a few years ago when people were getting first excited about ChatGPT. The gap has narrowed, but not by as much as people think.

AI produces output that is very convincing to a non-expert, and (dangerously), it's so good at looking like an expert, they might believe that it is an expert. But the moment you ask someone to use it for something they're an expert in themselves, the holes appear wide, consistent & obvious.

My favourite moment of seeing this in action was watching AI-worrier TV host/comedian Bill Maher. He has spent years talking about the dangers of AI taking everyone's jobs, destroying civilisation, ruining the economy, starting wars, "it's just getting better and better all the time", and so on. But one night he let slip a tell. "It's no good at writing jokes. Not yet, anyway". There you go, Bill... connect those dots...

There is real utility in it being a tool to help experts apply their expertise, as in this story where it speeds up some tasks to help the translator do part of the work, enhance their expertise, allow them to be more productive.

It's a better screwdriver, a better hammer, in the hands of somebody who knows what needs a screwdriver or a hammer. It doesn't replace them. It can't replace them. It's a tool that enhances the human, not an alternative.

I don't understand why this is not widely understood yet, but I'm sure it will in due course.

And I don't expect this to change. Even if the latest model scores 100% on every benchmark, all that really tells us is that it's now more productive/efficient than it was before at helping experts do that work, not that it can replace everyone in that category of work.

At what point does this become an issue for data quality and global epistemology?

It seems inevitable that we ask for more AI assistance on topics we don't understand. And therefore have the least context to correct. Result: a flood of poor quality information.

In areas we DO understand, we'll either not ask AI at all, or treat its results with a higher degree of skepticism. Result: a lack of high quality information.

Inevitably this means a higher volume of non-expert prompts gets translated into the next generation of internet content. AIs are pumping out more novice-level text and less expert guidance.

The result will be an internet full content written from the perspective of an ignoramus; not addressing any complex issues, staying surface level on every topic. Which will cascade into future models, etc.

> 2. AI is a terrible replacement for me - my skills are at such a high level that it’s almost theoretical that it’ll ever be good enough to replace me for 90% of what I get paid to do. It’s a tool at best.

Most? Perhaps it's depression, but I look back at my career and wonder if any code I've ever been paid to write is beyond what current AI can do.

Sure, this leaves me with the non-coding tasks of UX taste, and code review + a few other forms of QA (and, when self-employed, project management, game design, etc.), but man, I'm someone who actually learned to read in part on the Commodore 64 user manual (as in, trying to understand what PEAK and POKE meant concurrent with having "Jack and Jill go up the hill" picture books).

(And no, I'm not claiming LLMs make bug-free code, I see the bugs LLMs make during my code review of their output and some of them are awful, hence "this leaves me with …").

This is a new form of Gell-Mann Amnesia: https://en.wiktionary.org/wiki/Gell-Mann_Amnesia_effect

Reminded me of this post by EY. (You're making a different point about existing expertise, not LLM expertise, but I think it holds in general.)

Every month a new guy discovers LLMs; discovers a skill the current LLMs require to get good results; and writes about the future jobs that will always be available for smart people like HIM, that are SKILLED in using LLMs.

The next generation of AIs doesn't need his fancy prompt. The image model goes from needing to type in just the right set of weird words and cryptic sorcerous invocations, to most people being able to type in English what they want and get a pretty good result.

There are still tasks that require careful invocation. But they are a much smaller fraction of all the tasks people are trying to do, or you can get a bleh result without the elaborate invocation to get it really good. And to improve on the bleh result you need to be substantially more of an expert than back when the Guy was memorizing a rule about adding "trending on Artstation" to the image prompts, as would always require a human paid to do that.

Another generation of AIs comes out. The next generation of Clever Skills is obsolete. Image models just obey the instructions for compositing panels without mixing them up, and you don't need to be an expert to get them to do it right. Another human value-add is gone. A wider set of tasks require no human expert.

Now a new Guy notices LLMs have become useful in his field for the first time. He discovers they require SKILL to use CORRECTLY. He posts about how there will always be jobs for humans who are SKILLED in using LLMs like HIM.

But it is not an infinite cycle. It is not the same each time it repeats. Now the Guy is a highly paid programmer or a career mathematician in 2026, instead of a graphic artist in 2023.

In six months the models will no longer require his vaunted Skills.

And by then there will be another Guy.

But the process doesn't continue forever. The Guys are coming from fields that were harder and harder for AIs. The brief centaur eras are shorter and shorter.

Today it is writers who are laughing at how bad the LLMs are at their job, and who will perhaps soon be posting about how it takes Skill to get an LLM to do their job Correctly. But the models are coming faster, and the eras of kinds of human value-add in each field are shortening.

There is a point when you run out of Guys, either because the centaur eras are too short for people to develop SKILLs and post to Twitter about them; or because there are not lands left for AIs to conquer; or because ordinary people are not reassured by some Nobel laureate proclaiming there will always be jobs for Nobel laureates with the SKILLS to prompt robotized biology labs Correctly.

But we'll never run out of amateur economists who assert entirely without a brief contemporary example that there will always be jobs for humans skilled at operating AIs!

We'll run out of professional economists saying it when nobody is paid for that work anymore.

I guess we'll also run out of amateur economists when they're dead.

Source: https://x.com/allTheYud/status/2057136382817231151

My fear is in the future it won't matter. People will accept slop because while they can be convinced it's not as good as it could be, it's good enough. To them it's good enough because it's fast and cheap not because it's actually good. There won't be any room in the economy for the value human output brings because the economy will rearrange itself around AI and become completely dependent on cheap output, good enough or not.

Except that it is also quite difficult to assess the quality of a doctor or a software developer if you don't work in the field.

I've heard numerous cases where AI solved medical issues that doctor couldn't.

[dead]