> it just has to be established from the local side

This is exactly the problem. Unless you expect users to manually share their IPs with every other user in a given lobby through an external service, you would need to make a central peer discovery and connection coordination mechanism which ends up looking pretty similar to classic NAT traversal.