AI fan wikis riddled with errors are muddying the waters of games culture

So what's the solution? I imagine the right one isn't specific to fan wikis. We're going to need sites where 100% of contributed content is human-written. Who is building this? How will it work?

If the solution merely makes it high friction to post AI content that is probably good enough. In other words if someone has to copy/paste the output of an LLM into a text field and even then the tech detects that and throws an error that would be a big step in the right direction.

Reddit, Wikipedia, fan wikis, reviews, Hacker News comments - I'm sure there are a dozen more categories I'm missing.

In this new era it is almost zero effort to stand up a convincing website on any topic, which is mostly correct, but riddled with small confabulations and errors. It can take typing one sentence (even with grammatical errors) to do so, the bar is now zero for creating plausible 'content' and flooding the internet with it. As the article notes bad actors, but also people who have good intentions, are flooding the zone with slop.

If it was all clearly completely incorrect, it wouldn't be such a huge problem, but it is plausible and hard for humans to detect in a short time unless it is long form text.

Even search providers are giving us this new kind of generated result at the top of their results, which most people will be using blindly to verify answers. So it's turtles all the way down.

If you're not a domain expert you have no idea which things are right and which are wrong so it's very hard to determine the truth or trust results.

To top all that, LLMs are being trained on this new untrustworthy data without differentiating it from older data, and trained by other LLM instances in a sort of recursive loop to improve performance on very narrow goals. What could possibly go wrong!

I've seen a couple of these in search results while looking for info about more niche games and at first I thought they were fan made but like the article mentions they're not actually wikis, nor do they contain accurate information.

The ones I came across just contained nebulous prose with variously incorrect info about the game on a few barebones pages, with a custom domain name. They weren't using any wiki software.

I have uBlock Origin installed so not sure if the purpose was ads but I was baffled as to what the goal would be otherwise since they're useless for proper information, let alone a wiki.

I know the steel made before this particular year is look after because was not affected or build during periods of radiation, and that gives it qualities sought after.

When AI started I got a copy of 6 months before of wikipedia and open street map.

At some point you cannot trust anything because everything has been touched by everything as a regurgitation of everything.

Wait until you have to audit your average codebase post-AI.

AI is rotting pretty much everything that was carefully curated over the last couple of decades. If it can be expressed in bits it will be degraded. This is one of the most annoying side effects of AI, the general decrease in quality. Quality of discourse, quality in literature, quality of information, quality of software. It's everywhere.

I’m building a shed at the moment, and it’s become basically impossible to find reliable advice on tools or techniques by searching the web. It’s all just slop feeding back on itself now, unless you are lucky enough to find a forum post from before, say, 2023.

For example, try searching about what sort of nails you should be using to attach cedar lap siding.

On Google, you might find a few good results, but you’ll really need to scrutinize them to make sure they’re reliable. Most are slop on the ‘blogs’ of siding and roofing contractors.

On DDG, it’s even worse. Usually nailerguy.com ranks highly - it seems to be completely AI generated, complete with contradictions and impossible pictures. And, often, nailicy.com, a (seriously) nail salon in New Hampshire that I’m sure is confused about why all their web traffic isn’t translating into clients.

YouTube is about the only ‘reliable’ source for recent advice.

On the other hand, frontier LLMs are extremely good at getting super accurate info from games. They’ll happily disassemble the actual game binary to get to the bottom of certain game mechanics, including ones not obvious in the game itself, without even an explicit prompt to do so; I can merely ask questions about certain mechanics. That’s way better than often guessed (and guessed wrong) mechanics commonly found in game wikis. I’ve observed this over and over again when vibe coding some helper tools for some games I’m playing; I used to build these for wider communities and got some recognition for them, now I can do what I used to finish in 100 hours in like half an hour of prompting. I no longer share them though.

It’s the shit tier free or super low cost AI hallucination engines used by the masses that are the problem.

> Attempts to reach GameDB for comment via their public contact email resulted in an error, stating "the remote server is misconfigured".

this is such an emblematic thing for the slop peddlers

By the way, since fandom and fextralife were already such a huge blight on the concept of wiki, for a while people have been making solutions that might incidentally also help with AI slop wikis. For instance, https://getindie.wiki/

Fandom (ex-Wikia) has been a dumpster fire since day one.

> ... I put the question into Google and clicked the first result: the Nighthawks Wiki.

The problem here is Google. Bad websites have always been around.

People talk about their being a market for google without the AI features, but i think the real market would be google that filters out slop results.

Bonus points for also filtering out paywalls.

I'm tired of search results being useless. The signal to noise ratio on internet search is much worse now a days.

[flagged]