> Being able to reproduce the binary from the source code and being able to verify that it's the same as the original is quite important in some contexts

Why not build your own binaries and be done with that. If you don’t trust the compiler or the machine doing the build, just build the code yourself.