In my case sliding my card out of my wallet is faster than unlocking my phone given the lack of consistency of the fingerprint reader of my google pixel when using my smartphone case (and I am too clumsy to use a smartphone without a case covering both sides, broke too many screenw already). Some people just leave the card on their smartphone case too.
I also see a lot of people struggling because they need to pay while being on a call or because their smartphone is just way too big to be handled comfortably with one hand given the size of their hands.