alt Hacker News> All of the AI projects we have observed as a team are failing. Every single one – we have seen 0% success in a year and a half, not only amongst projects we have been asked to participate in, but even within projects that we have observed in passing while doing totally unrelated work.
That's got to be hyperbole, which blows out their credibility. They chose to say 'AI' rather than, for example, LLM, or Transformer model, or Diffusion model. This means they are including a huge swathe of things dating back to Expert Systems in their claim.
And who hasn't seen productivity gains from more established AI technology - at least things like semantic search? Who hasn't seen diffusion models generating content in roles that might have done the work by hand before? Who hasn't seen some kind of regression algorithm (even using linear regression in a supervised context counts as AI - so you can absolutely do AI even in tools like Excel) improve operation productivity?
Even if they narrowed it to the Transformer model LLMs which re-ignited recent public interest in AI, less ambitious projects to give them to engineering staff to automate easy but boring tasks in the background generally have been a success. More ambitious ones that are beyond what you'd reasonably expect the models to be able to do - for sure, those tend to fail. For most of these, the failure is predictable in advance, while some are at the boundary of what's possible, and so it is harder to predict (these are rationally genuine R&D projects).
Replies
> And who hasn't seen productivity gains from more established AI technology
You have to be very careful about claiming productivity gains. There may have been some instances of gains in a specific part of a workflow but does it slow down others or result in overall gains is yet to be empirically measured and validated. We’re seeing metrics like more lines of code, better unit tests, documentation, faster PRs etc. but the actual gains of businesses are still a question mark. Do more PRs lead to faster features being shipped or does it lead to slower reviews or bug ridden code that breaks user experience? I’ve see a lot of companies tout their metrics around more code being shipped but the same companies aren’t talking about how that translates to an actual dollar amount.
I think there's a big difference between individual employees using AI tools to boost their productivity - with things like Claude Code and Codex - and "AI projects" where companies build custom software on top of LLMs.
The former is easy to get right. Any software engineer (at least provided they aren't actively resisting the technology )can get useful results out of Claude Code these days.
The latter is really hard. LLMs are a strange beast to build software on, and most of the obvious projects - like the internal chatbots described in this article - are easy to have over-promise and under-deliver.
>semantic search
I'm doing fundraising for my tf-idf startup. It's named after a very big number!
I get what you are saying, and while I'll say it is definitely not 0%, I have seen very little in the way of useful software that is primarily generated. The vast majority just does not go the distance for whatever reason. I could explain many reasons, but I am getting really tired of explaining myself. If the tools were as great as everyone says, we'd be going through a software Renaissance, but we're not. I would argue a software dark ages since it feels like things are getting worse and I find bugs in what was historically very long running and stable applications. But, whatever. I think the author is clearly talking about modern AI, I don't think they need to be explicit about models.
Look, if they have data and it says 0%, and you have vibes that say that can’t be true, who should we believe?
Do you work with lots of companies and see large AI success stories?
Or do you just vibe that you personally find AI useful so it must also be a business success?
Look, I honestly don’t care, but I think “it must be false” is also unsubstantiated hyperbole. If an agency says they see no AI success, I see no particular reason to believe they’re lying.
They’re not saying AI can’t be a success. They’re saying they haven’t seen it. That matches my experience too. Proven AI success stories seem… vague, when you dig into the details, in my personal experience.
It doesn’t seem surprising to me.
Their previous article on AI shows a pretty strident ham-handedness. https://news.ycombinator.com/item?id=48002795
In general I find their submissions tend towards extreme grandiosity. I find I really appreciate people who have some nuance about the world, can see some duality, and the many many many submissions here are (I admit) often quite fun and enjoyable, but spoken much more from a bully pulpit perspective, with a zeal and self certainty that I find rarely coincides with truth-seeking.
I would love to learn about and from the details of their projects.
If you read the footnote, they follow up to say they've rejected 100% of the AI projects brought to them.
Go to their home page and one of their consulting selling points is recovering struggling projects.
One of their front-page selling points is that they use "ancient techniques" from books written prior to the year 2000, because presumably everything newer than that is bad?
> For non-executive management who might be struggling to deliver things that feel beyond their control, we have ancient techniques (see: books written between 1986 and 1999) to turn your team into the envy of the organisation, and we can drop in directly to get your team the resources it needs to save a struggling project.
This is entirely a selection bias issue that they've created for themselves: Advertise a consulting service for saving failing projects to companies that don't have internal expertise to handle it, then write blog posts that 100% of the projects you see are failing. Also refuse to help them, to guarantee they can't be converted to successful projects to keep the success number at 0%.