logoalt Hacker News

Symbiotetoday at 3:49 PM1 replyview on HN

Scrapers will ignore the dumps.

You can link to them in 10 million HTTP 429 responses, they will still ignore them.


Replies

ecetoday at 4:44 PM

Those scrapers can be blocked. Some scrapers won't ignore the responses, and maybe it'll lead to a meaningful reduction in scraping traffic.