If it was normal and expected that you carry the device around, to make purchases with, then all that would be very bad, and it becomes like a phone but worse in some ways (less ownership over it) and better in others (does not contain other personal data).

However, if it sits at home in a drawer, it can keep its camera on all it likes, transmitting images of darkness, and tell the bank repeatedly where your home address is, and sometimes (when in use) confirm what your face looks like. Not a privacy issue I think?

Probably it would become expected that you carry the thing around and it replaces cash and cards, but that seems to me to be the crucial step if it's going to have meaningful potential for spying.