logoalt Hacker News

spherical_cow • today at 8:57 AM • 0 replies • view on HN

Does Google scrappers honor robots.txt?