Yes. You can take advantage of Netfilter's flowtable infrastructure and if you have the right hardware (NVIDIA/Mellanox ConnectX-5 or MediaTekMT7621) it will actually offload the processing of these packets to the NIC hardware. This only applies to established connections, however, but that typically accounts for like 95% of the traffic passing through.