PS3 Emulator Devs Politely Ask That People Stop Flooding It with AI PRs

The problem is really behavioural, not the tooling. People that do not understand, test and document their decision making in their PRs should not be submitting them, regardless of what tooling (AI or otherwise) they used to create them.

This problem existed before AI, but it is now just worse due to the spamming nature of these "contributors". It's another form of endless September where people unfamiliar with the norms of team software development are overwhelming existing project maintainers faster than maintainers can teach them the norms of behaviour.

In the end, some sort of gatekeeping mechanism is needed to avoid overwhelming maintainers, whether it's a reputation system, membership in an in-group, or something else.

It’s starting to feel like we may need to go back to the model where you need to be invited to be able to submit code or PRs. The barrier is just too low now for popular projects.

I recently just started using Claude/ChatGPT/China models for some PS3 homebrew work.

Every model seemingly falls flat in this scope of programming. The PS3 is very complex and the tooling is fairly undocumented in a lot of instances. It doesn't surprise me most of these AI PR's are nonsense.

If anyone else has attempted writing PS3 homebrew apps using AI and has refined their tooling/systems/automation please let me know how you got the agents to work for you (:

We've seen a few takes on this kind of issue, but the solution I liked the best was the linux "developers take full responsibility" approach. The "Assisted-by:" tag was a pretty nice touch too.

The article unfortunately feels more like a rant than a good exploration of the problem space.

This is like when everyone started opening code of conduct addition PRs against every opensource repos except now a lot of these AI ones take actual effort to know it's in bad faith.

Or maybe it's worse because a lot of them aren't in bad faith they are well meaning people who just don't know or understand enough to realize they aren't being helpful.

I'm curious what percentage of PRs are just the AI blindly writing code and submitting a PR without testing, and which have at least been locally tested to some degree. Any OS maintainers have any insights on this?

If you look into the arcane architecture of the Playstation 3 console you quickly gain an appreciation for just how impressive the RPCS3 emulator is. PS3 is definitely one of hardest emulation targets, so it's wild they have the majority of the library working (with enhancements like upscaling and higher frame rates on many of the titles).

I guess it's nice people want to help and AI assisted coding can be fine but I can't imagine submitting a PR to a high-profile, much-revered project like that without reviewing and thoroughly testing it myself.

The emulation space is particularly bad about this because there are a lot of semi-technical and "well meaning" users who will do anything to get their games to play better and AI gives them a way to make it seem like they are doing something useful, without being able to judge the quality of the output they are producing.

One of the projects I work on recently had a guy drop by and explain that he wanted to use Claude to clean up our backlog and he absolutely could not fathom why I kept bringing up that we would only accept PRs that reduced our work instead of increasing it. "Do you know what Opus 4.7 is?" "Why are you so close-minded?". Unfortunately it is very hard for these users to understand that the thing they are using has a bar for quality and the bugs that still slip through cannot be solved by waving a magic wand at it.

what is the appeal of blindly blasting open source projects with high-volume PRs? If you're trying to help the project to accomplish something, it doesn't follow that a firehose approach is tenable, if only for the fact that reviewing the code takes time.

I just took a look at the RPCS3 PR history, and it doesn't look that bad. (Certainly worse than "no slop", but not what I'd call a flood.)

I went 10 pages back on GitHub, and the overwhelming number of PRs look like good PRs that have been merged. There's really only a single handful of rejected slop-looking PRs. (And another handful from a single user who seemingly didn't know how to use Git/GitHub and was turning local non-compiling commits into PRs somehow.)

I’ve read so many stories like this that I’ve actually gotten scared of making PRs open source projects.

There’s one in particular where a feature I really wanted didn’t exist, so I forked and had Codex 5.5 assist with building the feature on my local version. It works perfectly. My life has been improved in being able to have this feature now.

Normally I’d want to share it back with the community so others can benefit as well (presumably if I wanted this feature, others probably want it too.) But…I am not pretending this is perfect, great, or even good code. I spent about an hour total on it - it works, I haven’t had any issues with it, but it’s probably slop by any hard-core engineering account. And I neither want to get attacked for submitting slop nor do I have the time to properly engineer it to be hand-coded, so the net result is that it lives on my machine alone.

Is this the right outcome? I feel guilty that I’m getting a better version of this software and others aren’t. I want to help makes others lives easier too, but I don’t want to burden the project maintainers or get yelled at for submitting slop.

What’s the future look like here?

Regardless of whatever other opinions you have on this, I don't think its a fair characterization that the original tweet was "polite" or "nice".

why not just fork and forge an entirely new path unfettered by outdated norms? Isn't that the AI way?

I’m hopeful that in a year or so the models will be good enough to help productively with emulator development and that you will see a similar shift to these PRs that you did with security this spring.

Perhaps a "wall of shame" showing off bad PRs would make people think twice.

AI could be the end of FOSS if this doesn't stop. Why don't people get it? No AI means no AI.

Just paywall it, pays for development, queue prioritization and in return they get code merged into a popular project and visibility

Aww, PRs no longer open/welcome? Whatever will the usual suspects parrot now?

My personal schadenfreude aside, I wonder if this will follow a similar trajectory as security bug reports did recently. I'd be surprised, for a number of reasons, but the overall shape is looking awfully similar.