No but they can steal up to the pin-free amount of money your card allows. They can just hold a sumup to your pocket. Here in Europe most people use debit cards with non-reversible transactions and that limit is often 70-100€ which is quite a lot of money where I live. And they can steal a lot more if they follow you and shouldersurf your pincode.

I definitely use one of those wallets. They're quite convenient too.