How do you trust the results of a black box that are hard to verify? Most of us use proxies that are easier to check, like historical competence and maturity. For example, it's difficult to verify my compiler produces a binary that correctly implements the code I wrote. I can partially validate the result with testing, but when I send it out into the world to handle unknown inputs I'm trusting that the compiler has done its job correctly. I can have that trust because I know the compiler people (in aggregate) have thought very hard about correctness, and the project is mature enough that bugs are rare. If the GCC developers tomorrow were to start merging anything that passed the test suite and some other list of objective metrics, my trust would rightly decrease.