The noise is a huge problem, indeed. Still, a panel of review agents using models and harnesses different from the one implementing a set of changes has proven immensely useful for myself. The panel is basically an n×m matrix of agents and highly specific review prompts, i.e.:

- review for intent fulfillment: is the ticket done?

- review for correctness: race condition bugs, ...

- review for security: check against this list of sources and best practices

- review for api conformity: identify all surfaces of systems outside this codebase touched by the code changes and check against their docs

- etc. pp., same for maintainability, observability & analytics, test coverage, usage of feature flags

The matrix is sparse, so not every model is used with each of the review categories. Effort levels vary, too. The next stage does a consolidation across all findings, then another stage spins up one agent per finding and investigates the whole codebases for identical / similar instances of the finding; finally, it suggests a fix.

This works extremely well for finding deficits, but the amount of noise drives me insane, too. Lots of feedback is technically correct and "by the book", but pretty useless in practical terms – or even detrimental because the amount of code written and thus the size of the change set explodes. I'm not yet sure how to tackle this problem, any suggestions are welcome!