logoalt Hacker News

trollbridgeyesterday at 10:00 PM4 repliesview on HN

Yeah, it serves the purpose of blocking this kind of proxy traffic that isn't in Google's personal best interests.

Only Google is allowed to scrape the web.


Replies

1vuio0pswjnm7today at 4:49 AM

"Only Google is allowed to scrape the web."

If I'm not mistaken, the plaintiffs in the US v Google antitrust litigation in the DC Circuit tried to argue that website operators are biased toward allowing Google to crawl and against allowing other search engines to do the same

The Court rejected this argument because the plaintiffs did not present any evidence to support it

For someone who does not follow the web's history, how would one produce direct evidence that the bias exists

vachinatoday at 4:09 AM

This is demonstrably false by the success of many scrapers from AI companies.

show 1 reply
a456463yesterday at 10:47 PM

Yup exactly. Google must be the only one allowed to scrape the web. Google can't have any other competition. Calling it in "user's best interest" is just like their other marketing cons: "play integrity for user's security" etc

viraptortoday at 12:43 AM

Have you got any proof of Google scraping from residential proxies users don't know about, rather than from their clearly labelled AS? Otherwise you're mixing entirely different things into one claim.

show 2 replies