I have been a desktop trackpad user for so long now, I literally don’t want to use a mouse for anything except playing video games. The amount of flexibility offered by a good trackpad just wins most of the time as it is plenty accurate for quickly jumping around on one axis.
With a large enough trackpad, you could even move to a 1:1 type of movement, or add that functionality to a layer for the best of both worlds (like gyro enhanced aiming in games).
Wacom wants your money!
I’ve worked with some designers who did what you described with their huge tablets. Use the stylus to turn it into a giant touchpad. Works pretty good.