This aligns with something I've been noticing in practice: passing tests and being mergeable are fundamentally different quality bars. Tests verify behavior, but code review evaluates maintainability, readability, and whether the solution fits the broader architecture.

The SWE-bench metric essentially measures "can the AI produce a patch that makes tests pass" — which is closer to "junior developer who got the ticket done" than "experienced engineer who shipped clean code." The gap between those two is exactly where most code review friction lives.

What concerns me more is that as teams start using these benchmarks to evaluate AI coding tools, they might optimize for the wrong thing. A tool that produces mergeable PRs 40% of the time is arguably more valuable than one that passes tests 80% of the time but generates code that requires significant rework. We need benchmarks that capture the full review cycle, not just the CI pipeline.