logoalt Hacker News

londons_exploreyesterday at 9:23 PM12 repliesview on HN

We need more residential proxies, not less.

I've had enough of companies saying "you're connecting from an AWS IP address, therefore you aren't allowed in, or must buy enterprise licensing". Reddit is an example which totally blocks all data to non-residential IP's.

I want exactly the same content visible no matter who you are or where you are connecting from, and a robust network of residential proxies is a stepping stone to achieving that.


Replies

ndiddyyesterday at 9:44 PM

If you look at the article, the network they disrupted pays software vendors per-download to sneakily turn their users into residential proxy endpoints. I'm sure that at least some of the time the user is technically agreeing to some wording buried in the ToS saying they consent to this, but it's certainly unethical. I wouldn't want to proxy traffic from random people through my home network, that's how you get legal threats from media companies or the police called to your house.

show 2 replies
JDyeyesterday at 10:35 PM

I live in the UK and can't view a large portion of the internet without having to submit my ID to _every_ site serving anything deemed "not safe the for the children". I had a question about a new piercing and couldn't get info on it from Reddit because of that. I try using a VPN and they're blocked too. Luckily, I work at a copmany selling proxies so I've got free proxies whenever I want, but I shouldn't _need_ to use them.

I find it funny that companies like Reddit, who make their money entirely from content produced by users for free (which is also often sourced from other parts of the internet without permission), are so against their site being scraped that they have to objectively ruin the site for everyone using it. See the API changes and killing off of third party apps.

Obviously, it's mostly for advertising purposes, but they love to talk about the load scraping puts on their site, even suing AI companies and SerpApi for it. If it's truly that bad, just offer a free API for the scrapers to use - or even an API that works out just slightly cheaper than using proxies...

My ideal internet would look something like that, all content free and accessible to everyone.

show 3 replies
Aurornisyesterday at 10:04 PM

> I want exactly the same content visible no matter who you are or where you are connecting from

The reason those IP addresses get blocked is not because of "who" is connecting, but "what"

Traffic from datacenter address ranges to sites like Reddit is almost entirely bots and scrapers. They can put a tremendous load on your site because many will try to run their queries as fast as they can with as many IPs as they can get.

Blocking these IP addresses catches a few false positives, but it's an easy step to make botting and scraping a little more expensive. Residential proxies aren't all that expensive, but now there's a little line item bill that comes with their request volume that makes them think twice.

> We need more residential proxies, not less

Great, you can always volunteer your home IP address as a start. There are services that will pay you a nominal amount for it, even.

direwolf20yesterday at 9:33 PM

You can run one, something like ByteLixir, Traffmonetizer, Honeygain, Pawns, there are lots more, just google "share my internet for money"

What will you be proxying? Nobody knows! I haven't had the police at my house yet.

Seems a great way to say "fuck you" to companies that block IP addresses.

You may see a few more CAPTCHAs. If you have a dynamic IP address, not many.

show 1 reply
tokyobreakfastyesterday at 10:27 PM

> I've had enough of companies saying "you're connecting from an AWS IP address

I run a honeypot and the amount of bot traffic coming from AWS is insane. It's like 80% before filtering, and it's 100% illegitimate.

yuliyptoday at 1:30 AM

The end game of that is no useful content being accessible without login, or needing some sort of other proof-of-legitimacy.

show 2 replies
nine_ktoday at 12:27 AM

There's a company that pays you to keep their box connected to your residential router. I assume it sells residential proxy services, maybe also DDoS services, I don't know. It's aptly named Absurd Computing.

crtasmyesterday at 11:08 PM

I'm reading reddit.com from a Tor node, they also have a .onion domain you could use.

show 1 reply
packetslaveyesterday at 9:30 PM

> Reddit is an example which totally blocks all data to non-residential IP's.

No, we don't.

show 8 replies
xg15yesterday at 9:32 PM

Also, nevermind the tech companies building their own proxy networks, such as Find My or Amazon Sidewalk.

show 2 replies
a456463yesterday at 10:38 PM

This blog post from the company that used promise "don't be evil", one that steals water for data centers from vilages and towns via shady deals, whose whole premise it stealing other people's stuff and claiming it as their own and locking them out and selling their data.. Who made them the arbiter of the internet? No one!!!

They just stole this and get on their high horse to tell people how to use internet? You can eff right off Google.

BoredPositronyesterday at 10:03 PM

I still "run" a small ISP with a few thousand residential ips from my scraping days. The requirements are laughable and costs were negligible in the early 2000s.