alt Hacker NewsMy problem with these types of apps will always be that most software it interacts with will unfortunately be designed without accessibility in mind.
I've trialed a handful of these mouse-less programs before and I always run into the same hurdles.
E.g. where one or more of my apps have sliders which I can't interact with using clicks instead of drags (so I can't interact with it at all), or needing to click things like my MacOS dock, or even resize a window.
Replies
silon42 • today at 11:06 AM
What is needed is "Mouseless Friday" for UI designers/developers.
Hello, I'm the developer behind Neverclick. Neverclick doesn't use accessibility apis. It uses local computer vision so it works in every app.
I've had a poor experience with accessibility apis, they're clunky, slow, and unpredictable, and as you said many apps aren't built with accessibility in mind. With computer vision you don't have to worry about that.
This might sound strange, but the cv system in Neverclick actually runs significantly faster than UI Automation (which is the accessibility api on windows) in nearly all of my personal tests. I can't believe that accessibility apis are so poorly optimized that raw pixel analysis is faster lol. Also, I have users that use Neverclick on 10 year old hardware and they tell me that the cv runs instantly whereas UI Automation is super laggy for them.
Also, accessibility apis typically don't give you useful bounding boxes for text in text editors, this is another advantage of the cv approach.
I really wish the title of the post was "Desktop application for performing mouse actions with your keyboard using computer vision" so that it's more obvious that it uses cv, although it's my fault since the current title is how I have it in the readme and they probably just copied that.