And I'd add that the need for them is a sign they aren't worth doing.

As you say, worthwhile software is usually novel. And to justify our expense, it needs to be valuable. So to decide whether a project is worth doing, we're looking at some sort of estimate of return on investment.

That estimate will also, at least implicitly, have a range. That range is determined by both the I and the R. If you don't have a precise estimate of return, making your estimate of investment more precise doesn't help anything. And I've never seen an estimate of return both precise and accurate; business is even less certain than software.

In my opinion, effort put into careful estimates is almost always better put into early, iterative delivery and product management that maximizes the information gained. Shipping early and often buys much clearer information on both I and R than you can ever get in a conference room.

Of course all of this only matters if running an effective business is more important than managerial soap opera and office politics. Those often require estimates in much the same way they're required from Star Trek's engineers: so the people with main character syndrome have something to dramatically ignore or override to prove their dominance over the NPCs and material reality.

From another thought-experiment-y perspective:

Say you have problem A to solve. Then either one of those is true:

1) it has been solved before, ergo by virtue of software having a zero cost of copying (contrary to, say, a nail, a car, or a bridge), so there is no actual problem to be solved.

2) it hasn't been solved before, ergo it is a new problem, and thus at any moment you may turn a stone and discover something that was not foreseen (whether they are rabbits, yaks, bikesheds, dragons, or what have you eldritch horrors) and thus of unknown cost.

Any task that cannot be obviously fit into one or the other can nonetheless be split into an assembly of both.

Thus any attempt at estimates is as futile as gambling to win, tasks are only ever done when they're done, and "successful estimators" are kings of retconning.

It's all make-believe.

I am in a project where we have to give estimates in hours and days.

Needless to say we always underestimate. Or overestimate. Best case we use the underestimated task as buffer for the more complex ones.

And it has been years.

Giving estimations based on complexity would at least give a clear picture.

I honestly don’t know what the PO and TL gains with this absurd obscenity.

Spot on! Now add one more dimension (from machine learning projects and pretty much other research fields) -- you also never know if the result are going to even work.

Maybe the data just doesn't have the correlation you want. Maybe you just didn't try harder, never know for sure. But owners reasonably demand some estimations.

Another reason is that figuring out what the software to be written should actually do, and how it should work, is work that is part of the project and the time it will take needs to be estimated.

As well as the actual development work that will result, which isn't known yet at the time of estimation.

> If I ask you how long it'll take to add a new database field, and you've added a new database field 100s of times in the past and each time they take 1 day, your estimate for it is going to be very spot-on.

But in the software world, predictable and repetitive tasks are also the kinds of tasks that are most easily automated

The slow part isn't following a written "log in and run ALTER TABLE" runbook, it's the review of "does this solution make sense given what we're trying to do".

As I’ve gotten older, I find this to be untrue.

Estimating the unknown is itself a skill.

It’s like choosing to crash random parties everyday — at first, exceptionally novel but facing the unknown becomes itself mundane. Humans adapt. They build mental modals to approach the unknown and these mental modals are predictable.

I just don’t think most people get much practice.

"... anybody with any brains has already left town..." -- Bob Dylan

Anybody with any business experience has already isolated themselves from the certainty of software project failure, where "failure" is a euphemism for "late." So it doesn't matter if software can't be estimated.

This can be nerve-wracking to a beginner, but one gets used to it over time.

That's the point: how can you tell WHEN you are going to reach a place you've never been before traveling an uncertain path?

Making mistakes over and over again. And adding a lot of time buffers just to be safe

There is very little novel about most B2B CRUD and internal bespoke apps that most developers are doing. The novel part if any is implementing the business vertical logic

This is going to be immediately downvoted, but whenever I see software developers mention that estimating is uniquely challenging for software it immediately calls out a bunch of red flags. Estimates in software are challenging but no more or less challenging than other industries.

Typically the people that single out software as a unique snowflake in the world of delivery and estimation really just say out loud they have no experience in project management and no experience working outside of authoring code. The things that make estimations most challenging in software are the same factors that make estimations challenging everywhere else.

Repetitive and easy or worth automating are not the same. The fundamental problem is that a setup capable of solving any of the requests that come to it is as complex as just a whole programming language and then you’re back to square one.

Very insightful. This is also my go-to argument for why software engineering is real engineering.

As a project manager, it sounds like you're making excuses. Just give me a number, trust your gut!

We have a fundamental failure to communicate, what we're doing. The game project managers and finance believe we're all playing is a regression towards the mean, where everything is additive and rounds up to nice consistent formulaic sums. Whereas software development follows power law distributions. A good piece of software can deliver 10x, 100x, or 1000x the cost to produce it (ex: distribution, cost of delivering another copy of software is near 0 once written). You don't get that sort of upside with many other investments. Finance is happy with an NPV 8% above what they invest in. This means that when software people talk, everything they say sounds foreign, and everyone assumes it's because of jargon. It's not. The fish don't know they're swimming in water. When the fisherman comes, everyone is caught off guard.

So we get what the author talks about

> The estimates stopped being estimates. They became safety railings against being held accountable for unreasonable expectations.

We. Pad. Like. Crazy. Yes this is inefficient. Some project managers recognize this. We get theory of constraints. But rather than cull the layers of hierarchy that lead to the padding in the first place, all the blame for failure goes back to developers. Get hit on the head enough and you will stop acting in good faith and pad to save your ability to feed and cloth yourself.

I don't know how typical teams work, but in my case, new projects always come on top of other obligations. It may take 1 day to add the field, but how many meetings, fires, or other disturbances will happen during that day?

If software developers want to be then seriously as a profession, they need to be able to provide and justify estimates for their work.

Everything you said could apply to a new bridge, building, pharmaceutical compound, or anything else that is the result of a process with some known and some unknown steps.