Until last spring my bots never saw those Cloudflare CAPTCHAs despite not taking any serious evasion measures. Then all of a sudden they started getting them 100% of the time on sites I was crawling. For that particular project my webcrawler was cued by a bookmarklet, like I was picking the pages to import manually so switching to getting the data right out of the web browser got my system back on the road.
I always had the feeling that the Cloudflare CAPTCHAs discriminate in various ways, for instance I would see them much more when I was browsing on a Samsung Galaxy tablet than when I was browsing on an iPad. They disproportionately affect the disabled and I wouldn't be surprised other vulnerable populations.
If I had anything to do with it having a CAPTCHA or a GDPR popup would be an immediate WCAG fail at A level.
alt Hacker News
Until last spring my bots never saw those Cloudflare CAPTCHAs despite not taking any serious evasion measures. Then all of a sudden they started getting them 100% of the time on sites I was crawling. For that particular project my webcrawler was cued by a bookmarklet, like I was picking the pages to import manually so switching to getting the data right out of the web browser got my system back on the road.
I always had the feeling that the Cloudflare CAPTCHAs discriminate in various ways, for instance I would see them much more when I was browsing on a Samsung Galaxy tablet than when I was browsing on an iPad. They disproportionately affect the disabled and I wouldn't be surprised other vulnerable populations.
If I had anything to do with it having a CAPTCHA or a GDPR popup would be an immediate WCAG fail at A level.