alt Hacker NewsIf your product is used by humans, then it needs to be tested by humans - this cannot be automated. Those humans can be your QA people, or your customers. Perhaps your customers are happy to be testers, perhaps not.
Unit tests are very expensive and return little value. Conversely, a (manual?) 'smoke test' is very cheap and returns great value - the first thing you do when updating a server for example is to check it still responds (and nothing has gone wrong in the deployment process), takes 2 seconds to do, prevents highly embarrassing downtime due to a misconfigured docker pull or whatever.
Replies
> Unit tests are very expensive and return little value
Why are unit tests very expensive? This goes against everything I know.
Automated tests are no longer expensive to write and nowadays less expensive to execute.
If you think smoke testing is cheap, try doing it across concurrent releases and multiple envs, then see how fast the "just check it manually" step gets dropped when nobody owns it. Manual QA finds weird edge cases, but if that's all you have, regression will eat you alive.
Smoke tests and unit tests do different jobs, and calling unit tests low value is backwards unless your code almost never changes. If you skip automated tests because they are "expensive", you are betting release quality on hope and a lot of repeat work.
Sad to see you being downvoted, but you're exactly right. Well, almost - if you can afford to invest in a good integration test suite, that can catch many errors without requiring a human to regression-test every time.
At the same time, many quality attributes can't really be automatically tested, so automation shouldn't try to replace manual testing, it should be used to augment it.