logoalt Hacker News

ComputerPersontoday at 2:51 PM3 repliesview on HN

There are definitely patterns you can use against the scrapers. This maintainer just didn't have time for it, which is understandable.

We direct scraper traffic to a bot-specific server using Cloudflare's load balancer, slowly analyzing traffic and adding conditions one at a time. No accidental scraper DDoS in a long time.

Most scrapers are relatively honest in some way shape or form.


Replies

cogman10today at 3:33 PM

One thing that surpised me about gentoo is just how low budget it is as an operation. They are doing everything with a $12k budget. [1]

[1] https://www.gentoo.org/news/2026/01/05/new-year.html

show 1 reply
sporptoday at 4:18 PM

How did you implement this? My site's getting hammered, any tips would be appreciated

show 1 reply
Symbiotetoday at 3:46 PM

> Most scrapers are relatively honest in some way shape or form.

Did you miss a "dis" in there?