alt Hacker NewsI feel the pain — it’s very difficult to detect many of the less ethical scrapers. They use residential IP pools, rotate IPs, and provide valid user agents.
I feel the pain — it’s very difficult to detect many of the less ethical scrapers. They use residential IP pools, rotate IPs, and provide valid user agents.
And many proxy/scraping providers now use real browsers that can automatically bypass cloudflare captchas as well, and a bot with a real browser similarly won't be clicking on any invisible links, so... I am skeptical just how long this will make an appreciable difference.