alt Hacker News> Due to the limitations with browser and computer use capabilities, Saul could not post on platforms like Reddit and Product Hunt.
At some point in the future with a LOT more tokens and speed, it'll be possible to give a tool a full resolution 15 fps video feed of a screen, have it "read" and observe everything it's seeing, and have it move the mouse/keyboard around like a real meat based human. Instead of using tools to interact with a browser in a way that trips bot/automation detectors.
Replies
Eh, it’s not that different from what we have today and would likely just be a waste.
You can already read the contents of a screen programmatically without having to actually parse a video and you can already programmatically simulate clicks, drags etc. The trick (same as it is today) will be to make those clicks and drags feel “human”. Not too fast, not too slow, etc etc. But all those challenges exist today.
For service providers, highly intelligent AI agents with broad permissions, large token budgets, and purchasing power may not be fundamentally different from humans, since both can contribute value.