alt Hacker News> each LLM was tested 600 times, and in total over 10,000 questions and answers were assessed.
Okay but why do you feel 3 trials say as much as 10,000?
> each LLM was tested 600 times, and in total over 10,000 questions and answers were assessed.
Okay but why do you feel 3 trials say as much as 10,000?
>>Okay but why do you feel 3 trials say as much as 10,000?
I don't trust them so I've used 3 examples in incorrect questions/answers they have given and I got correct answers. I spend enough time with LLMs to know that if Grok answered it correctly and in detail then it wouldn't be a problem for GPT or Claude either.
The questions are also constructed in a way that it's easy to answer not fully (which they qualify as wrong). LLMs still answer them correctly and in detail though.