alt Hacker News> hundreds of requests per day
Does this matter? I can handle hundreds of requests per day with no issue on a home cable modem connection and my desktop pc running nginx. In fact I do and have since the 56k days. With an actual server or VPS with a big pipe in a datacenter this should literally be below noticing in terms of cost.
I would characterize this response to normal public website traffic as more harmful than the "problem". There's no need to be upset that web spiders are visiting your public website. That is what public websites are for.
Anyway, if you really do want to persue this silly thing start by looking up the ASN the IP is in and go from there. Don't rely on cloudflare to interpret the internet for you. I wrote an offline geo-ip and whois db dump world map visualizer in 2025 and these are the resources I use:
## RIR whois/peering db # RIPE NCC https://ftp.ripe.net/ripe/dbase/split/ripe.db.aut-num.gz # ARIN https://ftp.arin.net/pub/rr/arin.db.gz # APNIC https://ftp.apnic.net/apnic/whois/apnic.db.aut-num.gz # LACNIC https://ftp.lacnic.net/lacnic/dbase/lacnic.db.gz # AFRINIC https://ftp.afrinic.net/dbase/afrinic.db.gz ## RIR Delegation files # https://www-public.telecom-sudparis.eu/~maigron/rir-stats/ # https://ftp.afrinic.net/pub/stats/afrinic/delegated-afrinic-... # https://ftp.apnic.net/stats/apnic/delegated-apnic-extended-l... # https://ftp.arin.net/pub/stats/arin/delegated-arin-extended-... # https://ftp.lacnic.net/pub/stats/lacnic/delegated-lacnic-ext... # https://ftp.ripe.net/pub/stats/ripencc/delegated-ripencc-ext...
Replies
Yeah, I get hundreds of requests if not more per hour for some obscure personal but public servers that have ~0 legitimate other users. I guess once you're in some index that's just that. For an e-commerce shop, a few thousand irrelevant requests per day should just be part of the background noise that comes with being online these days? Cache is king.
> I can handle hundreds of requests per day with no issue on a home cable modem connection and my desktop pc running nginx.
And what kind of ecommerce site are you running on that nginx? First thing that get overwhelmed by bot traffic is DB. With a tiny one, with low total connection limit and bots hitting less common path like browsing 20th page of product search results, it is really easy to get DoS. I remember having to block Yandex user agent 20 years ago, surprising no one wanted to allocate additional resources so that crawler is happy.