alt Hacker NewsI guess they could have meant “cheat” as in swindle or defraud.
But, I think it is worth noting that the task is to make sure the paper gets a thorough review. If somebody works out a way to do good-quality reviews with the assistance of AI based tools (without other harms, like the potential leaking that was mentioned in the other branch), that’s fine, it isn’t swindling or defrauding the community to use computer-aided writing tools. Neither if they are classical computer tools like spell checkers, nor if they are novel ones like LLMs. So, I don’t think we should put a lot of effort into catching people who make their lives easier by using spell checkers or by using LLMs.
As long as they do it correctly!
Replies
Yes, that's along the lines of how I meant the word cheat.
I wouldn't specifically use either of those words because they both in my mind imply a fairly concrete victim, where here the victim is more nebulous. The journal is unlikely to be directly paying you for the review, so you aren't exactly "defrauding" them. You are likely being indirectly paid by being employed as a professor (or similar) by an institution that expects you to do things like review journal articles... which is likely the source of the motivation for being dishonest. But I don't have to specify motivation for doing the bad thing to say "that's a bad thing". "Cheat" manages to convey that it's a bad thing without being overly specific about the motivation.
I don't have a problem with a journal accepting AI assisted reviews, but when you submit a review to the journal you are submitting that you've reviewed it as per your agreement with the journal. When that agreement says "don't use AI", and you did use AI, you cheated.
My point is that LLMs, by virtue of how they work, cannot properly evaluate novel research.
Edit, consider the following hypothetical:
A couple of biologists travel to a remote location and discover a frog with an unusual method of attracting prey. This frog secretes its own blood onto leaves, and then captures the flies that land on the blood.
This is quite plausible from a perspective of the many, many, ways evolution drives predator-prey relations, but (to my knowledge) has not been shown before.
The biologists may have extensive documentation of this observation, but there is simply no way that an LLM would be able to evaluate this documentation.