I was careful to say "Moore's law-like". Moore's law is stated as "transistor count per area doubles about every 2 years." [0] As stated, then yes, this might be true but, while important, that's not really the quantity we care about.

As you point out it's really cost per transistor or cost per flop that we mostly care about. I'm finding it hard to find a succinct and clear plot, but I believe one is provided by Our World In Data on "GPU computational performance per dollar" [1] which, to my eyes, clearly shows exponential growth in computational power per dollar.

The picture for storage is a little more muddied but if you squint just right you might still be able to recover an uninterrupted exponential growth [2].

In my view, it's pretty clear that advances in AI have progressed so quickly because GPUs have been keeping up with the exponential growth of computational power (per unit cost).

Exponential growth in this area is usually characterized by "S-curves", where one technology gets saturated but the exponential increase in power or decrease in cost is picked up by another, adjacent, technology, that allows the growth to continue. For compute it's CPUs to GPUs. For storage it's platter drives that are now being overtaken by SSDs.

The more general phenomena is called Wright's law, or experience curve effects [3].

[0] https://en.wikipedia.org/wiki/Moore%27s_law

[1] https://ourworldindata.org/grapher/gpu-price-performance?ySc...

[2] https://ourworldindata.org/grapher/historical-cost-of-comput...

[3] https://en.wikipedia.org/wiki/Experience_curve_effect