alt Hacker News> anecdotally based on their own subjective experience
So the “subjective” part counts against them. It’s better to make things objective. At least they should be reproducible examples.
When it comes to the “anecdotally” part, that doesn’t matter. Anecdotes are sufficient for demonstrating capabilities. If you can get a race car around a track in three minutes and it takes me four minutes, that’s a three minute race car.
Replies
Studies have shown that software engineers are very bad at judging their own productivity. When a software engineer feels more productive the inverse is just as likely to be true. Thats why anecdotal data can't be trusted.
I have never once seen extraordinary claims of AI wins accompanied by code and prompts.
Anecdotal: (of an account) not necessarily true or reliable, because based on personal accounts rather than facts or research.
If you say you drove a 3 minute lap but you didn't time it, that's an anecdote (and is what I mean). If you measured it, that would be a fact.
The term "anecdotal evidence" is used as a criticism of evidence that is not gathered in a scientific manner. The criticism does not imply that a single sample (a car making a lap in 3 minutes) cannot be used as valid evidence of a claim (the car is capable of making a lap in 3 minutes).