And I'd add that the need for them is a sign they aren't worth doing.
As you say, worthwhile software is usually novel. And to justify our expense, it needs to be valuable. So to decide whether a project is worth doing, we're looking at some sort of estimate of return on investment.
That estimate will also, at least implicitly, have a range. That range is determined by both the I and the R. If you don't have a precise estimate of return, making your estimate of investment more precise doesn't help anything. And I've never seen an estimate of return both precise and accurate; business is even less certain than software.
In my opinion, effort put into careful estimates is almost always better put into early, iterative delivery and product management that maximizes the information gained. Shipping early and often buys much clearer information on both I and R than you can ever get in a conference room.
Of course all of this only matters if running an effective business is more important than managerial soap opera and office politics. Those often require estimates in much the same way they're required from Star Trek's engineers: so the people with main character syndrome have something to dramatically ignore or override to prove their dominance over the NPCs and material reality.
> As you say, worthwhile software is usually novel.
This is an interesting assumption. I’d argue that the overwhelming majority of software is the most boring LoB CRUD apps you can imagine, and not novel at all. Yet, people need to estimate the tasks on these projects as well.
< “Those often require estimates in much the same way they're required from Star Trek's engineers: so the people with main character syndrome have something to dramatically ignore or override to prove their dominance over the NPCs and material reality.”
This is so good.
The solution you described is basically agile, and that definitely includes estimates and deadlines.
The typical reason is that developers in large teams often don't know all the places where a field may be used - and their IDE could very well miss some. So automated testing or QA finds an issue, and then the unaccounted-for, time-consuming bugtracking begins.
CRUD: Create, Read, Update, Delete. Basically just inputting and displaying data from a database.