alt Hacker NewsAnubis[1] is a superb fix for sites not behind Cloudflare/Fastly/Bunny etc. We had millions of bot requests, on a site serving all countries so we couldn't block by country, with fake user-agents so we couldn't block using that. It uses 'proof of work' to detect real browser software.
Replies
I wish they'd ease up on the whole "don't change the logo without paying us" thing. The furry anime character is a turnoff for anyone with a brand or personal image that doesn't mesh with those subcultures.
Anubis sucks because CPU is cheap for scrapers and hard for humans.
I'm assuming a bot running a headless browser instance can still get past it?
It's still valuable to raise the cost of scraping of course. I don't think anything can really stop a determined scraper from impersonating a human. I wonder though if a system similar to Anubis but mining some crypto would make bots _welcome_ - since they're paying for their traffic.
Because of this wasteful crap the internet is so slow nowadays...
Try opening gcc bug tracker on your phone: https://gcc.gnu.org/bugzilla/
make your only legit users mine fake crypto to access your site, only costs them 5% battery on an android device
Which is trivially bypassed by an actual implementation of the proof of work in non-javascript, rendering it absolutely useless. The website is approximately 3800x times slower than native code, and hundreds of thousands of times slower than the CUDA kernel claude wrote. The "proof of work" is just non existent at that point, they're solved in milliseconds for what would take the browser version 10 minutes or more, it's security by obscurity being dressed up as something more.
pow_server http://127.0.0.1:8080 backend avx512-x16
──────────────────────────────────────────────────────────
uptime 00:03:12
solver ● BUSY difficulty 9, 0.3s
queue [####################............] 5/8 peak 12
──────────────────────────────────────────────────────────
accepted 1240 solved 1180
503 shed 48 504 timeout 2 4xx/5xx 10
──────────────────────────────────────────────────────────
last difficulty 5 nonce 645376 in 9 ms (101.6MH/s, avx512-x16)
hashes 3.90GH total avg 65.3MH/s
Ctrl-C to stop
It doesn't detect "real browsers". It simply adds some friction and is niche enough that AI scrapers have not bothered to bypass it yet.