Give your binary CAP_NET_BIND_SERVICE capability, and run it as a regular user. Almost no daemons need to run as root.