alt Hacker NewsI hate that these discussions go nowhere because there's no common metric anymore.
I have no idea what stuff like "is it always perfect?" means because it varies so much from person to person. Too many people have different expectations, are working on different problems, or have different standards or goals for there to be a common constructive discussion.
Totally agree that the lack of a common base of evaluation is terrible for the discussion, and benchmaxing only contributes to this.
The only way to get a sense for these systems is to use them on things you know well, and everyone knows different things at different levels.
People also tend to underestimate how fast this is moving and base their take on dated and subpar systems for a variety of reasons, a key one being that the firehose is too big for any one person to have a proper focus on all of it.