Can you?
I'm pretty sure if I try calling my bank or searching the website to confirm the developer's public key fingerprint, there's not going to be any answer. You have to ask Google's servers to give you the APK and trust what it gives you, either via the front-end called Aurora or the front-end called Play Store
Doesn't AppVerifier allow you to do just that?
Maybe not in practice, but in theory, it works. I don't think there's a better way of handling this without relying on some centralised authority (Google) to validate the authorship of an app, which is hardly desirable.