logoalt Hacker News

joshdavham10/01/20242 repliesview on HN

> can you write a web scraper in your browser? The answer is: YES, you can! So why is nobody doing it?

Completely agree with this sentiment.

I just spent the last couple of months developing a chrome extension, but recently also did an unrleated web scraping project where I looked into all the common tools like beautiful soup, selenium, playwright, pupeteer, etc, etc.

All of these tools were needlessly complicated and I was having a ton of trouble with sites that required authentication. I then realized it would be way easier to write some javascript and paste it in my browser to do the scraping. Worked like a charm!


Replies

moritzwarhier10/02/2024

Is Playwright really that complicated?

I feel that when it has been set up, it's very straightforward to use.

Maybe in contrast to other solutions you posted? Not sure about that though; having only brief experiences with both, Playwright seems like an improved Cypress to me.

metadat10/01/2024

You might like Tamper monkey. You can add a button to kick it off or whatever your heart desires.

Tampermoney also works around CORs issues with relative ease.

show 2 replies