alt Hacker NewsAnecdotally, I’ve found that codebases that enforce code coverage metrics often have worse behavior coverage than ones that don’t.
It’s a classic example of Goodhart’s Law in action. Code coverage metrics only measure what percentage of code the test suite causes to run. But it’s very, very easy to write tests that run code without actually confirming that it produces correct output for all possible inputs. And it’s very, very easy to assume that a module with 90+% code coverage also has 90+% behavior coverage, and then become complacent about reviewing the suite for proper behavior coverage.
Djikstra seemed like he was mostly against testing. But only because he was for proofs. A unit test is a single example. The real way to demonstrate the absence of errors is to prove they aren’t there (vis a vis axioms and assumptions).
But most developers don’t have the mathematical sophistication nor the time.
It’s not that unit testing is useless. Just good to know what their limitations are and to plan your testing strategy accordingly.