alt Hacker NewsThe trouble with estimation is that few places record the estimates and the actuals for future reference.
As I've pointed out before, the business of film completion bonds has this worked out. For about 3% to 5% of the cost of making a movie, you can buy an insurance policy that guarantees to the investors that they get a movie out or their money back.
What makes this work is that completion bond companies have the data to do good estimations. They have detailed spending data from previous movie productions. So they look at a script, see "car chase in city, 2 minutes screen time", and go to their database for the last thousand car chase scenes and the bell curve of how much they cost. Their estimates are imperfect, but their error is centered around zero. So completion bond companies make money on average.
The software industry buries their actual costs. That's why estimation doesn't work.
Replies
My employer does, but only analyses the overage, so when a project comes in under estimate they think it's because it went smoothly, the reality is actually that estimates only ever tell you how good you are at estimating.
Movie people are doing essentially the same thing over and over for 100 years. Everyone in the industry does things the same way. Their projects last only 1-2 years and are totally done once "in the can". In my experience this is almost never the case with technology based projects.
If your database of software projects has a thousand web browsers in it, you have to ask why all those programmers couldn't work together and write just ten or fifteen web browsers; maybe they're hobbyists who are reinventing the wheel for fun? If so, is that a good basis for estimating a new web browser?
The first time I worked on a database-backed web site, my team had written their own ORM, and while I was there we put together a thing so we could use the same view on more than one web page. The second time, we used web.py and no ORM. The third time, we used Ruby on Rails, which solved most of the problems we had on the first project I mentioned out of the box. Most recently, I used Django, which is roughly as convenient as Rails but easier to debug. I haven't tried it, but I'm guessing Claude Code could probably put together a usable Django site in a few minutes.
So achieving equivalent results requires very different activities over time, because each time someone does it, we accumulate not just knowledge but code that makes it easier the next time—thousands of times easier, if the novelty is small enough. This impedes estimation.
Often, too, the things we're trying to estimate are things like "add automatic Dart dependency support to Blaze". The time-consuming part of this is not breaking the rest of Blaze in the process. What reference class do we use for estimating adding new languages to Blaze without breaking it? Is Dart more like C++ or more like Go? These are questions for lawyers, not actuaries.
Maybe the videogames industry is different because they don't share code and don't maintain it? But it seems like an awful lot of games these days are getting written in Godot and Unity where 15 years ago people would have written their own engines... and problems that were major technical challenges for Quake are trivial now.
If someone wants ragdolls in their game, they can just license your code, right? They don't have to write it themselves. A dataset of how long it took people to implement ragdolls from scratch wouldn't provide any insight into how long it should take to integrate your ragdoll library.