alt Hacker NewsUnfortunately and you kind of have to count this as the cost of the Internet. You've wasted 500Mb of bandwidth.
I've had colocation for eight years+. My monthly b/w cost is now around 20-30Gb a month given to scrapers where I was only be using 1-2Gb a month, years prior.
I pay for premium bandwidth (it's a thing) and only get 2TB of usable data. Do I go offline or let it continue?
> You've wasted 500Mb of bandwidth.
Yep, it sucks, but on the positive side, I'm feeding 500Mb of garbage into them every day and that feels like enough of a small win for me.
> My monthly b/w cost is now around 20-30Gb a month given to scrapers [...] 1-2Gb a month
That definitely sucks.
> Do I go offline or let it continue?
Might be time to start blocking entire IP ranges and ASNs and see if that helps.