It also doesn't necessarily protect you from anything by obtaining it.

If you, say, go and get a certificate of no-permit needed when you're replacing all of your kitchen cabinets, and then you decide to move the sink while you're at it, you're back to still needing a permit for the plumbing change (and even if you aren't, if your neighbor calls the inspector and says they think you might be, the inspector might say, "no, we already know about that, it's just cabinets", or they might wonder if you are and then you're back in the situation you'd be in without the certificate).