Who gets to decide what is a good bot?
Presumably the site owner with Cloudflare providing enforcement.
Generally isn't a good bot one that respects robots.txt and is respectful of the site's resources by not being spammy?
by checking which follow robots.txt?
According to their plan, the good bots pay for scraping.
> Who gets to decide what is a good bot?
Paying Cloudflare's tolls => good bot.
Cloudflare, apparently.
Me… obviously
Presumably the site owner with Cloudflare providing enforcement.
Generally isn't a good bot one that respects robots.txt and is respectful of the site's resources by not being spammy?