alt Hacker NewsIt's mind blowing seeing these multi exaflop single rack systems.
The world's first exaflop supercomputer was Frontier. It was launched only 4 years ago in 2022.
It's not a fair comparison of course. FP4 in Helios barely qualifies as floating point. Frontier was proper fp64, 16 times the bit width and probably 256x as many transistors.
All the same just wow. Much compute.
Workloads did change over time. Back when we were first approaching practical exascale, the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision.
Now, the dominant compute-hungry workload is AI, where precision takes second place to the independent parameter count. To the point that the capacity of BF16, which were originally designed as a radical optimization for AI workloads, is sometimes considered wasteful now.
AI workloads have some truly peculiar and counterintuitive properties - the kind of things you might expect to see in biology instead of conventional computing. Intrinsic error tolerance, for one. It did necessitate some rethinking and reprioritization, and I'm not quite sure if we converged to the general shape of an "optimal" AI accelerator as of yet.