alt Hacker NewsLarge scale scraping tech is not as sophisticated as you'd think. A significant chunk of it is "get as much as possible, categorize and clean up later". Man, I really want the real web of the 2000s back, when things felt "real" more or less... how can we even get there.
Replies
n1xis10t • yesterday at 1:43 AM
If people start making search engines again and there is more competition for Google, I think things would be pretty sweet.
➕ show 3 replies
thethingundone • yesterday at 1:42 AM
I would understand that, but it seems they don’t store the stuff but recollect the same content every hour.
➕ show 1 reply
idiotsecant • yesterday at 5:52 AM
Have you ever listened to the 'high water mark' monologue from fear and loathing? It's pretty much just that. It was a unique time and it was neat that we got to see it, but it can't possibly happen again.
➕ show 1 reply
A curated web directory. Kind of like Yahoo had. The internet according to the dewey system with pages somehow rated for quality by actual humans (maybe something to learn from Wikipedia's approach here?)