> I would now have to make sure my IPv4 and IPv6 firewall stack are perfectly mirroring each other.

You'd still have that in your IPv4-with-more-bytes, as you'll still probably end up running dual-stack to address those old-v4-only sites. Or you'd do the same with v6 and run a tunnel to translate those v4-only addresses to your v4-with-more-bytes. So you're in the same situation either way.