I'd bet it depends on what they write to the LLM, since sycophancy is so strong in these things. If they write in a fairly agnostic way, it might default to sycophancy to you the dev, while if they write it in an opinionated way it might default to sycophancy to them. Regardless of who is right.

Yeah I've felt that gut punch all the time. Seems to happen with every technology. In the early days of GPS, people would ask for directions to the gas station then pull it up on their phone map and go "yeah, you're right!" So thanks for wasting my time, insulting my integrity, and killing my faith in humanity all in 3 words. And if you really need two opinions, it's so easy to just reverse the situation, "my GPS says there's an Exxon in two miles if I turn left here and then right, is that correct?"

I think I could qualify as an expert on various topics relative to many average people. But it always bothered me when experts wanted to "just be believed". In your example, it doesn't long to explain why a database is better than the file system in this instance and a moderately intelligent should be able to understand and not need you or Claude as the source of truth.

So I think it can be good people are able to "AI" to question expert opinions - though naturally, as you say, it's bad if people take the AI as a single authority akin to an expert.

>>I'm not against fact checking, but it absolutely felt like a punch in the gut coming from a non-dev. There was no "trust" there. She couldn't rely on my expertise, she had to go back to her "source of truth" (Claude) and asked for a second opinion on something that she wouldn't be able to verify.

This is probably how physicians feel when informed patients (whether using LLMs or not) ask questions (good or bad) during their patient encounters.

To be generous, it seems like the most natural way to phrase what happened. The non-developer punched in "should this be a database instead?" and it will say "You are right to question it! I was sloppy when .. blah blah blah blah blah"

I doubt the human went into a whole diatribe with the CLI about how the developer on the project with this many years was saying this or that but she wanted Claude to second guess them.

It's just saying "ah yeah, that went through without issue."

wow that wouldnt feel like a gut punch at all to me, it's normal for inexperienced people to not really understand what kinds of knowledge experts have or don't. I certainly come back from doctors appointments and google/gemini chat for whatever came up to get more detail. this person going to claude and getting immediate confirmation that you were exactly correct should have felt great, she would be like "wow, this guy's good, huh."

I can see how the situation is annoying there; I've had a similar frustration when doing manual code review for someone's obviously AI-slop PR only to have their bot auto-respond to my comments. There's an asymmetry of effort that feels unfair. Like, don't ask for manual review of something that contains lots of things that your own manual review should have caught first, and if I provide human feedback, I expect human responses.

That said, I wouldn't necessarily take it at face value that the person you were working with trusted the bot more than you— at the end of the day, the bot was doing all the implementation anyway, so all the interactions between her and the bot were probably framed in language like "what if we" or "I think it would be better if", so her saying "my programmer colleague thinks X" would naturally generate that "yeah your colleague is right, let's go that direction, makes sense" that she passed back to you, probably thinking it was a complement.

Maybe the real frustration is her not understanding the value-add that still exists there when someone with real expertise and more context can step in and provide that nudge in the right direction.

> Treating AI as if the output is factual is just a misuse of the tools, I think.

Especially because it's very often wrong. AI tools lack the necessary context of the specific situation and the surrounding technology environment to make appropriate suggestions for anything more complex than clean sheet designs and toy prototypes. There is too much complexity that is hidden in domain knowledge and not even exposed fully in documentation for any AI tool currently to ingest it all into its own context to give you an accurate answer, even setting aside hallucinations and that these tools are incapable of reasoning.

I have this same gut punch constantly. AI has destroyed functional outcomes in corporations, because managers are all deep in AI psychosis deciding that whatever the tool says is perfectly correct at all times. We've replaced analysis paralysis with a complete absence of thought from any human in high-level impactful strategy discussions, everything is entirely being abdicated to AI tools in so many companies. It's ridiculous.

AI will radically change the economy, but not because its replacing jobs, but because its eroding trust, domain knowledge and expertise, and thinking/creativity while being a massive waste of money and total boondoggle. This bubble can't burst soon enough so we can get on with doing shit that actually matters for people that actually helps them.

non-dev AI psychosis doesn’t bother me

I’ve seen how the market treats vibe coders that werent previoisly engineers. Coinbase’s CEO bragged about PM’s deploying code and everyone now thinks they are totally incompetent anytime a small UX problem occurs.

Engineers with prior experience now check a box, even if they are doing the same thing. Like a Compliance Officer or Cybersecurity professional checks a box

Annoyingly pedantically speaking, a filesystem is a type of database.