In case people missed it there's some additional important context:
- Major AI conference flooded with peer reviews written by AI
https://news.ycombinator.com/item?id=46088236
- "All OpenReview Data Leaks"
https://news.ycombinator.com/item?id=46073488
- "The Day Anonymity Died: Inside the OpenReview / ICLR 2026 Leak"
https://news.ycombinator.com/item?id=46082370
- More about the leak
https://forum.cspaper.org/topic/191/iclr-i-can-locate-reviewer-how-an-api-bug-turned-blind-review-into-a-data-apocalypse
The second one went under the radar, but basically OpenReview left the API open so you didn't need credentials. This meant all reviewers and authors were deanonymized across multiple conferences.All these links are for ICLR too, which is the #2 ML conference for those that don't know.
And for some important context of the link for this post, note that they only sampled 300 papers and found 50. It looks to be almost exclusively citations but those are probably the easiest things to verify.
And this week CVPR sent out notifications that OpenReview will be down between Dec 6th and Dec 9th. No explanation for why.
So we have reviewers using LLMs, authors using LLMs, and idk the conference systems writing their software with LLMs? Things seem pretty fragile right now...
I think at least this article should highlight one of the problems we have in academia right now (beyond just ML, though it is more egregious there): citation mining. It is pretty standard to have over 50 citations in your 10 page paper these days. You can bet that most of these are not going to be for the critical claims but instead heavily placed in the background section. I looked at a few of the papers and everyone I looked at had their hallucinated citations in background (or background in appendix) sections. So these are "filler" citations, which I think illustrates a problem: citations are being abused. I mean the metric hacking should be pretty obvious if you just look at how many citations ML people have. It's grown exponentially! Do we really need so many citations? I'm all for giving people credit but a hyper-fixation on citation count as our measure of credit just doesn't work. It's far too simple of a metric. Like we might as well measure how good of a coder you are by the number of lines of code you produce[0].
It really seems that academia doesn't scale very well...