If you care about performance you shouldn't be running from swap period. Swap is just a safety net.
Swap is required for hibernation. And with fast ssd or properly tuned zram/zswap one can run task that use 10%-20% more memory than installed RAM. Surely there is a slowdown, but it is not that big especially when SSD can read write more than 5GB/s.
It's OK to care about the performance of the safety net, too.