logoalt Hacker News

andailast Tuesday at 4:30 AM1 replyview on HN

Nice. I'm curious about Reddit scraping. Did you have to do anything special for that?

Also curious if you know anything about scraping Twitter.


Replies

backend_dev82last Tuesday at 4:33 AM

Looking into twitter right now, it seems like the way to do it is with headless browsers, but they usually cost money.

For reddit there used to be the json endpoints that you could just fetch, and you can batch your subreddits, so its nice end easy. They have just killed those...old.reddit still works, but i fear like the days are numbered there as well.