alt Hacker NewsWhile we're dealing with the same issue at work, I sometimes still wonder exactly who these scrapers are. OpenAI, Google, Anthropic and others are normally fairly well behaved (Minus Anthropic attempting to hide behind a browser-for-hire company). Mostly you can get IP range and user-agents for the large players, while it problems mostly stem from bots pretending to be Chrome.
Our largest offenders seems to be mostly limited to South-East Asia, so probably mostly Chinese AI projects, but that's speculation. I also don't recall ever seeing Grok IP ranges or a specific Grok UA, but that doesn't mean that they're hiding, perhaps they're just not interested.
Replies
Well, some websites claimed China is behind it, which could make sense (I would not know either way). At the same time, though, I kind of doubt your carte blanche here for all those companies. Why would you think none of them are responsible for the AI slop spam?
> Our largest offenders seems to be mostly limited to South-East Asia, so probably mostly Chinese AI projects, but that's speculation
Ok. So you also don't know. Well, I don't know either, but I don't make a speculation by claiming x, y, and z companies to be exempt. In my book they are all responsible.
Most of the nonsense comes through botnets/residential proxies, so it's very hard to say.