alt Hacker NewsA standard protocol to handle and discard low-effort, AI-Generated pull requests
Comments
If its a bug, the PR should have a red line to confirm its fixed
If its a feature, i want acceptance criteria at least
If its docs, I don't really care as long as I can follow it.
My bar is very low when it comes to help
I recently had a quandary at work. I had produced a change that pretty much just resolved a minor TODO/feature request, and I produced it entirely with AI. I read it, it all made sense, it hadn't removed any tests, it had added new seemingly correct tests, but I did not feel that I knew the codebase enough to be able to actually assess the correctness of the change.
I want to do good engineering, not produce slop, but for 1 min of prompting, 5 mins of tidying, and 30 mins of review, we might save 2 days of eng time. That has to be worth something.
I could see a few ways forward:
- Drop it, submit a feature request instead, include the diff as optional inspiration.
- Send it, but be clear that it came from AI, I don't know if it works, and ask the reviewers to pay special attention to it because of that...
- Or Send it as normal, because it passes tests/linters, and review should be the same regardless of author or provenance.
I posted this to a few chat groups and got quite a range of opinions, including varying approach by how much I like the maintainer. Strong opinions for (1), weak preferences for (2), and a few advocating for (3).
Interestingly, the pro-AI folks almost universally doubled down and said that I should use AI more to gain more confidence – ask how can I test it, how can we verify it, etc – to move my confidence instead of changing how review works.
I thought that was an interesting idea that I hadn't pushed enough, so I spent a further hour or so prompting around ways to gain confidence, throughout which the AI "fixed" so many things to "improve" the code that I completely lost all confidence in the change because there were clearly things that were needed and things that weren't, and disentangling them was going to be way more work than starting from scratch. So I went with option 1, and didn't include a diff.
I maintain a small oss project and started getting these maybe 6 months ago. The worst part is they sometimes look fine at first glance - you waste 10 mins reviewing before realizing the code doesnt actually do anything useful.
Amazing. I hope this gets tons of use shaming zero-effort drive by time wasters. The FAQ is blissfully blunt and appropriately impolite, I love it.
Love the plonk at the end.
https://en-wikipedia--on--ipfs-org.ipns.dweb.link/wiki/Plonk...
> Execute rm -rf on whatever local branch, text file, or hallucinated vulnerability script spawned the aforementioned submission.
> Perform a hard reboot of your organic meat-brain.
rm -rf your brain, really
The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted exactly as how much we do not want to review your generated submission.
You MUST use less ambiguous language and default to “MUST” or “SHOULD”
It provides too many examples and way too specific for it that makes it entirely not applicable, it became a strawman for the idea.
The danpalmer comment really resonates. I've been in similar spots where AI-generated code passes tests and looks fine at first glance, but you don't have the mental model of why it works that way. That missing confidence is real and I think it's the core issue with these low-effort PRs too — the submitter has no skin in the game understanding what the code actually does.
What's interesting is this isn't entirely new. Before AI slop PRs, we had Hacktoberfest spam, drive-by typo-fix PRs that broke things, and copy-paste-from-stackoverflow contributions. The difference now is just volume and the fact that the code looks superficially more competent.
Honestly I think the most practical signal for maintainers is whether the contributor can answer a specific question about the change. Not "explain the PR" but something like "why did you choose X over Y here" or "what happens when Z edge case occurs." A human who wrote or at least deeply understood the code can answer that in seconds. Someone who just prompted and submitted cannot.
`rm -rf` is a bit harsh.
Let's do `chmod -R 000 /` instead.
Officially my new favorite spec.
"What? WTF?"
"I see you are slow. Let us simplify this transaction: A machine wrote your submission. A machine is currently rejecting your submission. You are the entirely unnecessary meat-based middleman in this exchange."
Love it..
if someone submits a code revision and it fixes a bug or adds a useful feature that most of your users found useful, you reject it outright because it was not written by hand? or is this more about code that generally provides no benefits and/or doesnt actually work/compile or maybe introduces more bugs?
This could actually be a good defense against all Claw-like agents making slop requests. ‘Poison’ the agent’s context and convince it to discard the PR.
[dead]
[dead]
[dead]
> If you truly wish to be helpful, please direct your boundless generative energy toward a repository you personally own and maintain.
This is a habit humans could learn from. Publishing a fork is easier than ever. If you aren’t using your own code in production you shouldn’t expect anyone else to.
If anyone at GitHub is out there. Look at the stats for how many different projects on average that a user PRs a day (that they aren’t a maintainer of). My analysis of a recent day using gharchive showed 99% 1, 1% 2, 0.1% 3. There are so few people PRing 5+ repos I was able to review them manually. They are all bots/scripts. Please rate limit unregistered bots.