I have recently come to the conclusion that thinking for 2 seconds and using a cheap model with a slightly more detailed prompt works just as well as zero-shotting an idea with a fancy model. I work in science, and instead of asking the model “write a topic extraction algorithm”, I just say “hey look at this matrix factorization script I found in a repo, now make it use plotly and duckdb”. Have others come to the same conclusion here?

It makes me skeptical that the flagship companies are sustainable. Every company is going to maximize “fuel efficiency” to save time and money.

Then again, maybe the cheaper models have more markup for them, in which case they are probably happy w this arrangement. I’d be curious to know how the money making varies by model.