logoalt Hacker News

20ktoday at 2:04 AM1 replyview on HN

Google's web scraping functionality has been acting as a ddos for more than two decades. I've seen literally hundreds of reports of them attacking websites and taking them down, where there's nothing you can do but accept the traffic, or get delisted

This is unfortunately nothing new. There's no correct way to tell them to fuck off, they do not care, and they never will do. People have even taken them to court over this


Replies

weird-eye-issuetoday at 5:21 AM

If a site cannot handle traffic from the real Googlebot that is a serious issue with the site itself since it's actually pretty conservative

Also I should note there are lots of fake Googlebots...

show 1 reply