logoalt Hacker News

hurturuelast Friday at 10:40 AM5 repliesview on HN

in general the consensus on HN is that the web should be free, scraping public content should be allowed, and net neutrality is desired.

do we want to change that? do we want to require scrapers to pay for network usage, like the ISPs were demanding from Netflix? is net neutrality a bad thing after all?


Replies

johnethlast Friday at 1:23 PM

I think, for many, the web should be free for humans.

When scraping was mainly used to build things like search indexes which are ultimately mutually beneficial to both the website owner and the search engine, and the scrapers were not abusive, nobody really had a problem.

But for generative AI training and access, with scrapers that DDoS everything in sight, and which ultimately cause visits to the websites to fall significantly and merely return a mangled copy of its content back to the user, scraping is a bad thing. It also doesn't help that the generative AI companies haven't paid most people for their training data.

komali2last Friday at 11:37 AM

I'm completely happy for everything to be free. Free as in freedom, especially! Agpl3, creative commons, let's do it!

But for some reason corporations don't want that, I guess they want to be allowed to just take from the commons and give nothing in return :/

wrxdlast Friday at 11:38 AM

The general consensus here is also that a DDOS attack is bad. I haven't seen objections against respectful scraping. You can say many things about AI scrapers but I wouldn't call them respectful at all.

show 3 replies
WhyOhWhyQlast Friday at 10:42 AM

If net neutrality is a trojan horse for 'Sam Altman and the Antrhopic guy own everything I do' then I voice my support for a different path.

dns_sneklast Friday at 11:20 AM

Net neutrality has nothing to do with how content publishers treat visitors, it's about ISPs who try to interfere based on the content of the traffic instead of just providing "dumb pipes" (infrastructure) like they're supposed to.

I can't speak for everyone, but the web should be free and scraping should be allowed insofar that it promotes dissemination of knowledge and data in a sustainable way that benefits our society and generations to come. You're doing the thing where you're trying to pervert the original intent behind those beliefs.

I see this as a clear example of the paradox of tolerance.

show 1 reply