logoalt Hacker News

mpegyesterday at 6:04 PM1 replyview on HN

There are reliable ways of differentiating human from cheap, bulk scraping bots.

But if the bot is advanced / expensive enough, it gets a lot harder. Where this product's market sits is in giving a paid way to access content compared to having to spin up bots that run js, from real IP addresses, etc. all of which are more expensive


Replies

xur17yesterday at 6:27 PM

Agreed. To me this feels like the perfect solution for websites and ai crawlers. Instead of having crawlers paying proxy services and captcha solvers, they can pay the website itself. As a web scraper, I'd happily pay the website provider to get access if it meant easy access to the content. Heck, as a human, I'd pay to avoid the dumb captchas.