You can maybe connect to everyone over IPv4, but chances are that that path is strictly worse (in terms of latency, P2P reachability, congestion et.c) than a v6 one would be.
For example, two IPv6 peers can often trivially reach each other even behind firewalls (using UDP hole punching). For NAT, having too restrictive a NAT gateway on either side can easily prevent reachability.
I have tailscale on all my mobile/portable devices I use away from home. It punches holes so I don't have to, even makes DNS work for my tailnet in a way I've never been able to get to work the way I want the normal way.