logoalt Hacker News

CDRdudeyesterday at 7:27 PM3 repliesview on HN

Isn’t it somewhat likely that a lot of shady crawlers pretend to be a google bot with their user agent?


Replies

ipaddryesterday at 7:33 PM

If you check the ips they belong to Google.

Plus if you run adsense google with ignore crawler rules and visit the page from google ips and from some shady ip. Wonder if it is the same for sites using Analytics.

cookiengineertoday at 12:12 AM

Gotta dig -x to find out if they really are from Google

bfleschyesterday at 7:53 PM

Why you still have the idea in your head that they play by the rules. With the current administration they have been empowered to extract maximum value from us.

In the early days of smartphone use, Google and Facebook uploaded contact lists of every single smartphone user to their servers.