alt Hacker NewsI will no longer upload to NEW
Comments
While I, like almost everyone, doesn't love reading LLM output, code review is an excellent use for LLMs. You know what I like less than reading LLM output?
- Slow PR review
- Nitpicks on style, not substance
- Async waiting for review or re-review after making changes, harder to stay in what passes for a "flow state" nowadays
- A lot of a ceremony around straightforward changes
We use LLM reviews at my company and it's caught things human reviews have not (as in in the early days we human-reviewed code and then did tests running the LLM afterwards and it caught things we didn't). It can review code at any time of the day and re-review it just as quickly as you can address the feedback.
I predict we (my company) are less than a year out from having a fully automated pipeline for reviews with only humans getting called in when it touches certain parts of the code. We already have a crude system for detecting certain file paths that we want a human to review if touched and we tweak that list over time as needed.
I do not understand why people keep building these anti-human LLM systems.
It just needs ca 5 seconds to realize "oh damn, I would not want to be at the receiving end of this" and conclude that it's probably a bad idea.
Why does this keep happening?
Especially since the idea of having such automated reviews probably isn't bad. But why mess up the execution like this basically every single time?
Respects maintainers time: DSFG uses LLM to automate reviews. A human reviewer reads the LLM report, the submission, and replies with the review.
Does not respect the maintainer's time: DSFG uses an LLM to automate reviews. It auto-replies to submissions. There is an appeal process to theoretically handle errors.
The crux of polite LLM use is on whom the burden of the flood of generated output falls.
If you are the consumer of your automated output, all good. Say you generate a report and read it, the burden of inaccuracy or verbose output generated in seconds falls on you. Not only does this avoid wasting other people's time, you have the power to go fix the tool to stop wasting yours.
If the output goes directly to others, the burden of correcting inaccuracy falls on them. The burden of reading it falls on them. And they usually then have to scale some wall of automation to reach a human and convince them to fix it. Users subjected to this will be rightly pissed off.
I don't have a problem with reading LLM feedback on my code, to check if there's anything it pointed out that's valid. Sometimes it points out things that aren't valid, and I ignore it. Sometimes it points out things that are valid, and I change it.
I don't feel the need to make someone else read the LLM output first just to do an inferior job than what I would do (since I know the code and can more easily identify what is valid vs what isn't).
Related: https://www.debian.org/vote/2026/vote_002
GR resulted in Proposal E, Choice 5: Responsible Use of Generative AI. Clearly stating: "Any such automated process should be overseen by a human who remains accountable for its behavior and output."
i wozld not care for most other actors even in the FOSS universe to swallow their pride and put LLM into their workflows. However, Debian proved in countless occasions before that they can and will go the route that is right, not the one that is easy, even if it makes everything slow or cumbersome and I think thats a good thing and the reason why so many others build their stuff in top of it.
The company I work for has an automatic review system as a first pass.
It became actually useful only with the advent of Opus 5 and its competitors. Nowadays it usually makes a few good points, but in instances where it doesn't, it's quite the stubborn asshole.
This one time I had to paste a function from a library because it couldn't get it to its digital head that its assuming the function has a certain implementation based on its name only.
> spend my valuable developer time (this is my free time)...
Lesson for us all is that if we do not own the thing/project/etc, things can suddenly change without our decision, and all time we put into it might become a waste-of-effort retroactively
OP does not understand where the bottleneck is. It is not in contributors pushing NEW packages, but in the reviewers' time.
They can stop, it does not matter, there will be others ready to contribute.
It is a ruthless reality but you have to preserve the most precious resource in FOSS which currently is maintainers.
I think some developers are kinda dense or something, and don't understand that shoving AI content in people's faces is simply rude.
In recent versions of iTerm2 for example, when using the plain old menubar and hovering over some items, panels appear showing awful sloppy AI images. WTF? https://files.catbox.moe/5ivkhw.png - That makes me question everything... Was the developer always tasteless? Or did they get a taste bypass?
Obciously the solution is to fight fire with fire. Spin up your own LLM and have it respond to every single point. Exhaustively. Have it ask semi-reasonable questions that must be answered by the humans.
I wager the resulting usage would drive the system costs up to the point that it would be dissembled and decommissioned. That only happens if the people behind it feel the cost personally.
This is a very relatable sentiment. On the other hand, checking code is one of the best use-cases for an LLM, it is better at checking code than generating it.
It seems like the question is where to fit it in to the pipeline. The team responsible for reviewing submissions could leverage it to speed up the process, and still respond personally.
So how often is this LLM wrong?
This is a strongly phrased message but the problem could be solved with an opt-out.
Even something as simple as writing the phrase "No LLMs please" somewhere in the upload, that if present, prevents LLMs being triggered. Nothing elaborate and it preserves the default-on behavior the team seems to want.
I think it is false to equate this to just two options:
a) spend my valuable developer time (this is my free time) on reading machine
output which indeed might be completely wrong and get angry and upset for
reasons which have been listed in great detail by those opposed to LLM-use in
Debian on debian-vote
b) not read the machine-generate review and have the resources that were
required to generate it not only be wasted, but let my upload become another
item in a statistic which shows how many tokens Debian is using up because the
more tokens we use the better our software, right?
The author wrote so much, but it has so little substance. LLMs can provide valuable feedback, but they can also produce garbage. Given that the author didn't show any bad outputs and built their message around "I don't like reading it", it sounds like a personal problem that should have stayed in their head.
I don't think there's another way. Everybody is being spammed with Gen AI outputs and the only even to it seems like a paradox way to fight it is with LLMs of some kind or go around it like what the EU is trying to do with watermarking and then detect AI and have different processing for it. I kinda think we could end up there, but we are still not there.
I’m still sad that Debian is now integrating (external?) LLM in the build process which seems antithetical to the core values of Debian (as pointed out).
There is no alternative to Debian for me. I’m ready to justy throw out all my computers and become a goat herder at this point.
PS I’ll remove memory and storage from said computers to pay for land and goats before I chuck them out of my window.
I'm disappointed by how things like this are (as far as we know) implemented. There could've been a clear page with reasoning and stats from backtesting showing the system is actually useful. There could've been a good framework for this review that makes it clear and succinct. I'm actually a big fan of automated early reviews, which are really good these days.
But it looks like the team decided to shoot themselves in the foot there and both implement and communicate the change badly...
As somebody who's actually a proponent of providing an AI-review before human review, I think making AI review optional could be a solution.
MD in the thread has a very agreeable view:
> I do not believe that it is reasonable to ask people to spend time on an unreviewed LLM output. If it was too much work for the DFSG team to read it then it is for me as well. I do not object in principle to the use of a LLM, but the DFSG team needs to take full responsibility for its output: this kind of disclaimer is not acceptable.
Once again, human attention and human effort, etc
Elsewhere in the email thread:
Picking at random a successful and unsuccessful LLM-generated review from the dashboard, the successful review consists of 734 words (~2.5 printed pages) and the unsuccessful review weighs in at a whopping 1,535 words (~5 printed pages). I haven't tried to fully read either report, nor spend brain cycles checking if the LLM output is legitimate or not. I just want to emphasize the many orders of magnitude change in review size that's being generated, and that uploaders may need to spend time trying to review and understand in this new process.
A bit rude to expect people to swallow 5 pages of slop to figure out why review failed.
EDIT: made my comment more constructive, but originally asked if any distro has adopted a hard anti-AI stance.
There's no point volunteering free time just to deal with slop.
have two queues. one is the normal one with the AI review and all that, the other is for AI-abstainers, where:
> I rather wait a year before my package makes it through NEW (yes, that happened) than have my strong opposition to LLMs be ignored and all my uploads being automatically screened without my consent.
so those entries go in the "abstain-from-LLM-queue", where there is no guarantee of timeliness at all, and human volunteers can process that queue manually as they are available
tldr just tag the entries as "OK for LLM" or not. it's just like organic and non-organic produce, it can be done.
If whoever approves or rejects the package wants to use an LLM to help with the submission, it's their thing. Not for, not against, LLMs can be good "interns" a lot of the time.
That said, the LLM presence and involvement should be confined to the team using it. "We don't know what it says, it is not a part of judgement, it may make mistakes, but here's a wall of text from it for you to read for who knows what benefit" is quite a dick move. Don't throw slop into people's faces like that.
[flagged]
[dead]
Divas!
I'm not anti-ai but I am firmly anti-make-people-read-llm-output-I-didn't-read-myself, and this breaks this.
They also have a reward system that punishes people for ignoring the llm review (by pushing their submission lower in the queue).