logoalt Hacker News

hollowturtleyesterday at 8:41 AM2 repliesview on HN

Ehm what would stop ai scrapers from using a browser like a normal user would? Google bot already does, it can execute js and can read spa client side generated content, so it proves can be done at scale, and I'm pretty sure some ai scrapers already do


Replies

dirkcyesterday at 2:45 PM

If you decrypt the content on the client side using an expensive decryption algorithm the scraper needs to spend the computing resource to decrypt.

show 1 reply
vivzkestrelyesterday at 10:27 AM

rate limit per ip that progressively keeps decreasing req/mins every few mins?

show 3 replies