NemID, the previous national 2-factor solution, used a small card with rows of pre-printed single-use codes. When you logged in to a bank or a public sector website, it would ask for a random code at a specific row and column number. Once the system registered that you had just a handful of codes left, a new card would be sent to you via snailmail. It worked fine for the time.
The current system, MitID, depends on smartphones, though you can get an an external key generator as a backup too.
Yeah but functionally it is the same. If the website is down it doesn't matter if I got the OTP code from a piece of paper or the dongle.
The big drawback of one time passwords is that it doesn't protect against man-in-the-middle attacks such as phishing, which is in practice one of the most common attacks on systems of this scale.
The logistics operation involved in distributing codes is also very expensive and inflexible. You may need to authenticate payments a dozen times in an hour one day, when you are on a farmers market which doesn't take card payments or you are out dining with friends, and another day not at all.
Given all this, a good old public key infrastructure makes sense. But that is unfortunately also usually the first step to a complexity explosion.