alt Hacker NewsI use Playwright to intercept all requests and responses and have Claude Code navigate to a website like YouTube and click and interact with all the elements and inputs while recording all the requests and responses associated with each interaction. Then it creates a detailed strongly typed API to interact with any website using the underlying API.
Yes, I know it likely breaks everybody's terms of service but at the same time I'm not loading gigabytes of ads, images, markup, to accomplish things.
If anyone is interested I can take some time and publish it this week.
Replies
I also do this. My primary use case is for reproducing page layout and styling at any given tree in the dom. So, capturing various states of a component etc.
I also use it to automatically retrieve page responsiveness behavior in complex web apps. It uses playwright to adjust the width and monitor entire trees for exact changes which it writes structured data that includes the complete cascade of styles relevant with screenshots to support the snapshots.
There are tools you can buy that let you do this kind of inspection manually, but they are designed for humans. So, lots of clickety-clackety and human speed results.
---
My first reaction to seeing this FP was why are people still releasing MCPs? So far I've managed to completely avoid that hype loop and went straight to building custom CLIs even before skills were a thing.
I think people are still not realizing the power and efficiency of direct access to things you want and skills to guide the AI in using the access effectively.
Maybe I'm missing something in this particular use case?
Why even use Playwright for this? I feel like Claude just needs agent-browser and it can generate deterministic code from it.
Yes, please do and ping me when it's done lol. Did you make it into an agent skill?
Would this hypothetically be able to download arbitrary videos from youtube without the constant yt-dlp arms race?
I just ask Claude to reverse engineer the site with Chrome MCP. It goes to work by itself, uses your Chrome logged in session cookies, etc.
Very interested. Would even pay for an api for this. I am doing something similar with vibium and need something more token efficient.
I was doing similar by capturing XHR requests while clicking through manually, then asking codex to reverse engineer the API from the export.
Never tried that level of autonomy though. How long is your iteration cycle?
If I had to guess, mine was maybe 10-20 minutes over a few prompts.
Commenting to follow up.
Please do!
Please publish!
isnt it what everyone that needs web validation does?
[dead]
I love how HN is loving this idea when it's the exact same thing Anthropic and OpenAi (and every other llm maker) did.
It's God's gift to them when it lets them bypass ads and dl copyrighted material. But it's Satan's curse on humanity when the Zuck does it to train his llm and dl copyrighted material.