alt Hacker NewsHow is this a racket? This is a service website owners want, and it (that is, Cloudflare’s resurrection of the 402 Payment Required response) seems to be one of the few schemes that can work at scale. The current situation, where AI companies benefit from content created under the premise of advertising revenue, is not just unethical, it’s uneconomical to the point of driving content creators out of business.
Replies
jychang • yesterday at 9:32 PM
Yes, I agree here.
Everyone should remember, limitations of technology is not meant to define society. Instead, we build edge cases into technology to better match society’s general expectations.
A website owner saying “yes normal humans, no bad bots, EXCEPT good bots” is totally fine.
Didn't they turn this on by default?
If websites owners truly wanted it, it would be a 'do thing to opt in' and everyone would rush to that.
Now I do think this kind of thing is good for many reasons, but I also see many reasons this can be problematic (that I did not consider the first time I read about it).
I myself would prefer an option to throttle the bots, and give them a 'you can spider at 2am-5am once per month access' via robots.txt, header or something..
you come more than twice in a month and get blocked or pay for access to static version hosted on other server / cdn..
best of both worlds without some of the negative issues.
Otherwise it's a play that helps cloudflare more than anyone else, and hurts more than [open][other][AI] - etc. imho.