alt Hacker NewsCan somebody tell me what is a normal "cost of doing business" level of bot traffic these days? I have way too much bot traffic like everybody else, but I don't know if I am an outlier or just run of the mill. I get about 100k bot hits a day, presumably because I have about 350k pages on my site.
Esports vertical: I get about 5-20b bot hits per day (unwanted; includes both IA, brute forcer, "security" scanners, wp-admin/ requests), 1.5m google spider (search; respectful of crawl delay), and about 50-100m human (largely mobile).
For unwanted bots I serve incorrect information -- it's online gaming match history without much text so requests flagged as unwanted bots will, instead of heavy database queries, get plausibly random numbers -- seeded by the user so they stay stable -- KDA, win/loss rates, rankings.
A few dozen million distinct pages but they are numeric stats for user profiles, match stats with little to none paragraph form of text.