alt Hacker NewsYes, but not in the “dump another chunk of all written language in the bucket and stir”-sense which is what bitter lesson became synonymous with.
That may not be the intent of the original article, but over the past few years that’s what the phrase turned into.
Replies
theptip • today at 2:53 AM
The bitter lesson just means “compute scaling beats hand-tuned architectures in the long run”.
As GP said. More RLHF is in fact the bitter lesson.
GPT-6 is supposed to be using a much larger base model that just finished pretraining so the "dump another chunk of all written language" approach is still going strong.