David Patterson is such a legend! From RAID to RISC and one of the best books in computer architecture, he's on my personal hall of fame.

Several years ago I was at one of the Berkley AMP Lab retreats at Asilomar, and as I was hanging out, I couldn't figure how I know this person in front of me, until an hour later when I saw his name during a panel :)).

It was always the network. And David Patterson, after RISC, started working on iRAM, that was tackling a related problem.

NVIDIA bought Mellanox/Infiniband, but Google has historically excelled at networking, and the TPU seems to be designed to scale out in the best possible way.