logoalt Hacker News

empikolast Saturday at 4:52 PM1 replyview on HN

AI generated reviews are a huge problem even at the most prestigious ML conferences. It is hard to argue against them, since the weaknesses they identify are usually in well formulated, and it is hard to argue that subjectively they are not that important. ACL recently started requiring Limitations section in their paper where authors should transparently discuss what are the limits. Unfortunately, that section is basically a honeypot for AI reviews as they can easily identify the sentences where authors admitted that their paper is not perfect and use it to generate reasons to reject. As a result, I started recommending being really careful in that particular section.


Replies

birn559last Saturday at 7:27 PM

Wow, that's a terrible second order effect with very real impact on the quality of publications.