I guess I'm looking for wireguard's version of STUN. And now that I know what to google for, finally found some promising leads.

https://github.com/jwhited/wgsd

https://www.jordanwhited.com/posts/wireguard-endpoint-discov...

https://github.com/tjjh89017/stunmesh-go