> it still has to punch through your router's firewall
That's why most routers use a stateful firewall. Then nothing has to "punch through" it just has to be established from the local side.
> block all incoming IPv6 traffic unless there is outbound traffic first, and provide little to no support for custom IPv6 rules.
This is why STUN exists.
> my bet is that there are close to zero popular games that actually use true peer to peer networking.
For game state? You're probably right. For low latency voice chat? It's more common than you'd think.
The complication starts when such an ephemeral port gets connection from somewhere else, which is the crucial part not the creation of such ports. That is not supported necessarily by firewalls, or not that simple than just having a stateful firewall.
> it just has to be established from the local side
This is exactly the problem. Unless you expect users to manually share their IPs with every other user in a given lobby through an external service, you would need to make a central peer discovery and connection coordination mechanism which ends up looking pretty similar to classic NAT traversal.