logoalt Hacker News

ipeevtoday at 4:37 AM0 repliesview on HN

I think my reaction is mostly puzzlement. I can see a sensible point or several in the article, but I was not always sure how big a point the author was trying to make.

At the narrower level, it seems to be saying that benchmarks are easier to interpret when you know what they really are. That makes sense. If a circuit is known to be a multiplier, that tells you more than if it is just called `c6288`.

That is also why I thought of Python benchmarks. In something like `pyperformance`, names such as `json_loads`, `python_startup`, or `nbody` already tell you something about the workload. So when you compare results, you have a better sense of what kind of task a system is doing well on. But so what? It is just benchmarks. They don't guarantee anything about anything anyway.

What made it harder for me to follow was that this fairly modest point is wrapped in a lot of jokes and swipes about AI and corporate AI language. Some of that is funny, but it also made me less sure what the main point was supposed to be. Was the article really about benchmark interpretation, or was that mostly a vehicle for making a broader point about AI hype and technical understanding?

So I do think there is a real point in there. I just found it slightly hard to separate that point from the style and the jokes.