alt Hacker NewsIt's a trivial example. This won't be useful outside of a VERY specific domain without more parameters. Many people need to know about the bitter lesson.
https://en.wikipedia.org/wiki/Bitter_lesson
Over time, I'm sure we'll be able to filter information better and get parameter counts down, but I wouldn't count on that within the next 6 months.
Replies
alightsoul • today at 4:58 PM
I think the bitter lesson only talks about task performance but not computational efficiency. Could tiny models improve efficiency? Maybe by just using a general architecture on specialized data, so the artichecture itself is not task specific?
The point here is that the library's overhead cost is very low. The fact that a tiny model can reach 10M tokens per second means that the overhead of token decode, memory allocation, calling the model, etc. is very low. The model doesn't actually need to be useful to prove that point.