alt Hacker NewsSO the point of this is ... what exactly?
A big arrow to an interface element which ... has a label that explains what it does?
So...a label for a label?
Replies
Finally we've built the reverse centaur from Cory Doctorow's classic novel "Don't Become the Reverse Centaur"
It's so your agent can make a big arrow on the screen saying "click this, human" so that it can keep doing things
I ran two experiments which require the `claude` CLI tool be installed.
For the first experiment, Claude stepped me through how to edit its settings.json to remove warnings from its configuration: a little clunky as I had to respond to Claude's CLI prompts to get it to auto run.
In the second experiment, Claude walked me through how to use Apple's Compressor.app to speed up a video. It walked me through dragging the file into compressor and clicking on the appropriate UI elements to do this.
So, one use case for the bigarrow skill enables Claude to step users through a relatively complex UI to accomplish a task.
There are likely more efficient ways to accomplish the tasks at hand (e.g. ffmpeg for the second experiment), but this use case is interesting if using a GUI tool is required.
Glad you asked. Pointing my aging aunt to the right place on the screen to click without having to take control of her laptop, of course.
It seems silly but I think there's a good use case for this:
Helping people deal with bad UX.
Everybody who has worked in a traditional corporate environment has encountered poorly tutorialized, poorly labeled, counterintuitive user interfaces where the only way to know how to get something done is to have somebody sit over your shoulder and tell you which buttons to click.
Oh, you want to figure out which OIM entitlements your coworker has that you don't to debug an access issue? You just need to know that you have to click the "compliance" button to get there. Why is it called "compliance" when what it actually does is let you export comparisons from the user tables? Because fuck you that's why. Memorize it.
An AI assistant that can read the procedures for you, or sometimes even read the source code, or the website code, can show you how to accomplish something that the UI on its own fails to do.