Publish fake stats, get cited in LLMs

> In cartography, a trap street is a fictitious entry in the form of a misrepresented street on a map, often outside the area the map nominally covers, for the purpose of "trapping" plagiarists of the map who, if caught, would be unable to explain the inclusion of the "trap street" on their map as innocent.

https://en.wikipedia.org/wiki/Trap_street

I'm reminded of a section of Cory Doctorow's latest non-fiction, The Reverse Centaur's Guide To Life After AI (it is of course an excellent read), where he discusses how the semantic web and having an internet set-up for agents and AI to use won't work:

>It also hasn't happened because there is a reward for cheating. Scammers will label their phone numbers as the customer service number for your cable operator or bank. Counterfeiters will label their batteries or phone cases as official accessories. It's hard enough to get people to correctly label their stuff—but fact-checking billions of labels created by millions of people, none of whom you know personally or whose identities you can verify? That's insurmountable.

There's more he says—the preceding paragraph is below—but that line "there is a reward for cheating" is something I keep thinking about. It's hard not to see in so many of the discussions of LLMs.

>For one thing, there are irreconcilable differences in the ways in which things should be described. A site run by transphobes isn't going to label every story about a trans woman as "a story about a woman," irrespective of what the standard says. Microsoft is never going to label Excel as equivalent to Apple's Numbers, Google Sheets, and LibreOffice's Calc. Hermès is never going to list a $40,000 handbag as being made of the same material and having the same capacity as a $30 leather handbag of the same size and shape.

This is a known problem, when ChatGPT uses web search tool, top 10 results are often LLM generated sources as well.

They now train on their own content.

Eating your own d̶o̶g̶ f̶o̶o̶d̶ poo.

This has existed for a long time before LLM, although in a slower form. On Wikipedia it's called "citogenesis".

LLMs are facing the same problem pre-google humans faced:

Is this information on this website true/high quality?

We need a google for LLMs!

Oh, but LLMs don't consume ads.

We need ads for LLMs!

Help.

This problem is so bad that you can create a website just to review your own website so when LLMs want to pull out reviews for the website the fake reviews count

This is insane.

How can this be fixed? Nothing prevents you from publishing straight BS, and LLMs will use any site that doesn't have a garbage reputation as a valid third-party source.

Of course, LLMs already use Reddit, Facebook and co. as a source, so it doesn't really matter...

See Chavda’s paradox https://zencapital.substack.com/p/chavdas-paradox

This is fine work, we need more communication to the general public about this

> Its made-up numbers have also ended up in an academic paper and in a bank's investment research.

It's an excellent way of showing how unreliable the information you find online is (and it will only get worse in the future), but willfully creating misinformation and made up shit and offering it as fact feels hostile to humanity.

Surely nobody got hurt as a consequence of those "facts", but bad decisions with real world impact could be made regardless. It feels like the behavior should be shunned in a general case (the same way I'd shun those numerous spam sites that pollute search results and very much would prefer they not be allowed to exist), even if in this case the intent was good.

Problem is, you can't really do anything about it, other than figuring out a way to take down hostile sites and keep them from popping up, so it's even worse than fraud or even just bad research in academia.

I suspect there are already many fake stats sites like this by the time I read this.

I wonder why did Google derank it. Did it correctly identify it as LLM written and sink it because of that?

this is actually insane. and people still use chatgpt to generate reports for their managers. snowball of fake data

AI doesn't know or care if the data is fake.

What's the conclusion? Publish fake content (or not)?

If I'm not wrong ChatGPT indexes on Bing, curious if the site and stats rank well on Bing

"You really think someone would do that? Just go on the internet and tell lies?"

I thought everybody knew this was a problem. Scrapers vacuum up data but don't have any way tell what's true or false.

It's interesting to have a specific case where obviously false data is being spread, but there's nearly an infinite number of more subtle variations around.

Nine out of ten stats are made up on the spot.

don’t

if you just lie and an llm or a newspaper or a kid believe you, at that point it isn’t a criticism of their naivete, you are just lying

similarly hacking through systems or picking locks isn’t a showcase of weakness if systems, it’s just being a hacker

[flagged]

[flagged]

[flagged]