alt Hacker NewsI think these "Flash" models are sort of an evolutionary dead end. Sure, there are some routine tasks and applications where they can be used. But for the actual novel development work? It's much better to run a big model at high power for 30 mins than watch the Flash model struggle for 2 hours and produce massive churn.
Same reason your phone has a few big CPU cores for real work, it's much better to "race to idle" than have an "efficient" core struggle. Shitty experience, shitty power efficiency.
Replies
It depends how you use the models. These small models work great for developers who prefer to stay more in the loop, and only task the model with things that can really only be interpreted in one way.
Not to mention, they’re great for self-hosting and getting yourself to not be dependent on some API that can go down or be altered at any time.
Big models seem to mostly be good for pushing ahead the frontier - the smaller models tend to gain the frontier’s capabilities after only a handful of months anyway. Many are perfectly content remaining a few months behind the bleeding edge.
If you have good feedback signals, like tests/benchmarks/etc, then it is potentially better to do multiple turns where model uses that to adjust code. Which might not need as smart a model.