logoalt Hacker News

mzajctoday at 5:43 AM0 repliesview on HN

My cgit instance is incredibly friendly to bots; I go out of my way to support git's fast "smart" HTTP protocol, and the clone URL is linked on every project's root page. I even included the full clone command on 429 error pages.

The end result was that I got millions of requests daily trying to download every HTML page (every variation of action, commit hash, branch, whatever else cgit allows) and basically no traffic to the proper clone route.

Turns out the bots just don't care about your bot-friendly access method and will instead throw more illicitly-proxied residential IPs at the problem.