Is this basically the bitter lesson applied to the very act of creating model weights? Absolutely wild, and the implications for RSI are noteworthy.
This goes against the bitter lesson I think.
This goes against the bitter lesson I think.