That's exactly right.

I hav set up a system where customer success and sales can drop in artifacts of customers talking about what they value (emails, transcripts, etc) and skills analyS them and then use them to add context to issues in the backlog.

The idea is that everything in the backlog is tied to an explanation of who it benefits and how it benefits them. We're using AI to merge multiple sources and automate the writing of it. The hope is it streamlines that communication. Our backlog issues now are 3-4 pages that explain very clearly why the issue matters, what it's higher level goal is, etc.

At first engineering was like "woa that's a lot of text" but after reading it was then "that's the best written issue I've ever seen".

Okay, so cool we are streamlining product management and setting ourselves up to automate customer feedback to development pipeline, dramatically cutting down on that issue discernment bottleneck you're pointing at...

..except today I found an issue with critical hallucinations in it. It mixed up what the customer said and what the cs rep said, to the extent that the issue was just straight up incorrect. This was with Opus 3.7 extended thinking. (Mind you it was a big transcript and pushing the limits of context window, loading multiple skills, etc)

So there's some serious potential, but it's just not there yet. Even if all this works flawlessly, the context these models can hold at once is like 0.1% of what a human can (if not less). So we will still need the humans for quite a while to make the harder decisions.

This is in a very leading edge startup pushing the limits of what LLMs can do... And even in this context optimized for LLM success it's still no where close to replacing people. We get a ton of value out of LLMs, but let me clarify that the hold up isn't just fact checking, it goes way beyond that.

In some ways I keep thinking it comes down to context management. Humans can hold so many orders of magnitude more context. Context is the bottleneck. The tech is a long way off being capable enough, and even when it is, there will be lots of operational and cultural obstacles to getting the right context into the AI.

And then there is the jevons paradox consideration...

It feels like we are a long way off. It seems plausible a generation from now employment will look very different, and I can kind of grasp how we get there, but I'm extremely skeptical of any unemployment apocalypse on a 5 year time horizon being triggered by AI. Maybe an unrelated economic shock, but not AI.