logoalt Hacker News

solenoid0937yesterday at 4:09 PM1 replyview on HN

https://marginlab.ai/trackers/claude-code-historical-perform...


Replies

taylorfinleyyesterday at 6:54 PM

Surely they are testing their optimizations against common benchmarks internally? I bet the "real world task" degradation is larger by some multiple than it appears when measured through a benchmark that is part of the target.