alt Hacker NewsThere is always a huge discrepancy between benchmarks and real world capabilities.
For my use case deepseek 4.1 flash is almost as dumb as haiku 5.5. Both are fast garbage producers.
Replies
embedding-shape • today at 6:45 PM
> There is always a huge discrepancy between benchmarks and real world capabilities.
I mean of course, the real world is messy, the models are only good for what they've been trained on, and post-training frequently takes benchmark scores into account too, leading to this sort of "Great score, sucks in the real world" syndrome that seemingly most models are affected by.
I was beginning to worry I had overlooked the inner beauty of DeepSeek. When I have tried it, it was totally useless. Never understood HN cheering for it.