> ipv6 packet does not have any port field

Yes, that's why I said I know it was mixing of layers.

However ports are a layer violation in a strict sense, introduced as a workaround because there was no easy way to just add thousands of new IPs to a single host back in the IPv4 days. No need to continue a workaround that causes grief on a daily basis.