logoalt Hacker News

Tcepsatoday at 2:24 PM0 repliesview on HN

Presumably it captures a screenshot and then uses CV on the resulting image to "see" the application and identify the various things that you might want to click.