A100: 312 TFLOP/s for FP16
but it is very impressive how far modern CPUs get as well (also in smart phones!)
Intel Xeon 6980P: 128 cores x 1024 FP16 FLOP/cycle/core x 3.2 GHz: 419 TFLOP/s
alt Hacker News
Intel Xeon 6980P: 128 cores x 1024 FP16 FLOP/cycle/core x 3.2 GHz: 419 TFLOP/s