Then they should use an appropriately scoped API, as OP suggested.
Controlling my computer is appropriate scope for an accessibility tool
Controlling my computer is appropriate scope for an accessibility tool