logoalt Hacker News

klipttoday at 7:36 PM1 replyview on HN

Perhaps an analogy to Moore's law?

Bitter lesson #1: don't waste time optimizing code when a faster processor is around the corner.

What countered it: Moore's law stopped working.

Bitter lesson #2 similarly relies on scaling laws that might have diminishing returns wrt model runtime vs intelligence. Runtime matters for turnaround on the problem you're solving.


Replies

wild_eggtoday at 8:05 PM

Moore's Law has nothing to do with processors getting faster. Dennard scaling stopped working but Moore just slowed somewhat, not stopped.