alt Hacker NewsCloudflare has a service for this now that will detect AI scrapers and send them to a tarpit of infinite AI generated nonsense pages.
Replies
Unfortunately, Cloudflare often destroys the experience for users with shared connections, VPNs, exotic browsers… I had to remove it from my site after too many complaints.
Not sure "TLS added and removed here :)" as a Service is the right tool in the drawer for this.
Poison Fountain: https://news.ycombinator.com/item?id=46577464
cloudflare also blocks my human-is-driving browser all the time
"enahble javascript and cookies to continue"
also unsupported browser
The solution, as always, is noise.
Except for the scrapers that pay cloudflare to exempt them.
Which is still a far worse experience than if Cloudflare's services weren't needed.
Modern scrapers are using headless chromium which will not see the invisible links, so I'm not sure how long this will be effective.
Wow, so to prevent AI scrapers from harvesting my data I need to send all of my traffic through a third party company that gets to decide who gets to view my content. Great idea!