alt Hacker News>If scrapers reuse a cookie, traditional anti-abuse mechanisms will work.
Scrapers can't change their strategy? It's not hard to think of mitigations: keep on using a session until you're banned, and failing that rotate your IP. Nowadays you don't even need to think. You can probably tell a LLM "write me a scraper that has adaptive anti-anti-bot mechanism".
Replies
bulder • yesterday at 10:35 PM
Plenty of these abusive scrapers are utilizing retail residential proxies, which will be applying forced rotations to avoid "burning" their compromised and or otherwise surreptitiously utilized IP address.
> Scrapers can't change their strategy?
Yes, but you normally don't try to fight all strategies up front, you add stuff on a need-to basis. Currently, lots of people are in the stage of "need to get this thing using bunch of IPs to stop hammering my servers", apparently Anubis helps them with that. If that stops working, other tactics are tried, and so it goes on for the life of your service. Hasn't stopped yet, doesn't seem to be stopped soon either.