With IPv6 it's plausible that you can avoid collisions as long as you use an expensive hash function, but for v4 how do you avoid IP collisions?

With only 22 bits of entropy in your v4 addresses, you'll get accidental collisions with only ~2000 users.