logoalt Hacker News

cowartclast Friday at 3:07 PM0 repliesview on HN

The scarcity framing assumes compute is the bottleneck. For most production deployment's Ive seen, the actual bottleneck is evaluation and knowing what to trust.

You can throw cheaper models at a problem all day but, if you can't measure where the model fails on your data, You're just making mistakes faster at a lower cost.

Compute gets cheaper. Reliable evaluation doesn't.