logoalt Hacker News

cor_NEEL_iustoday at 11:38 AM0 repliesview on HN

The validation point is real. We tested this with AI presentation tools specifically - gave 6 of them the same prompt and fact-checked every claim against primary sources. Best accuracy was 44%. Most were under 20%.

The pattern was consistent: the tools produce confident, well-formatted output that looks thoroughly researched. But more than half the statistics were either distorted or completely fabricated. The worst part was finding the same fake stats appearing across multiple tools - not because they independently verified anything, but because they all absorbed the same bad data from training.

The productivity gains from AI are real, but so is the validation cost. People just aren't accounting for it.