alt Hacker NewsI run scripts on my servers on an hourly basis to check which are the top 25 IPs visiting the server (aggregated by /24). If anyone in those top 25 IPs are from China, Vietnam, etc, or from Alibaba/Amazon/etc, the /24 gets blocked by iptables.
It's far from perfect, but it was a quick way to get rid of bots, while not completely blocking people from countries such as Vietnam.
However, on a Gitlab instance I manage (500 users), we have to restrict viewing of git logs and pretty much everything except issues. The bots were too aggressive. Chinese crawlers have access to a huge range of IPs and they often do only 10-20 requests per day, while generating in total over 50k requests per day. Our server load went from 99% down to 0.1% after that (and it's a fairly big server).
My solution was similar. Anything coming from the ASN of a major cloud provider gets a CAPTCHA with a little nuance to allow Google and Bing to index. That seems to do a pretty good job. Also, anything coming out of China or Singapore also gets a CAPTCHA as my site is not popular in those regions and many Chinese bots seem to show up as a Chinese mobile provider. So far, the bots have never attempted to solve the CAPTCHA.