> Can you access their websites without the need to confirm 'who you are' with their app?

Yes, none of them required me to use the app a single time. In fact, for all the banks I work with, I always identified myself at a local office when opening the account for the first time, the last one less than a year ago. And all of them allow me to operate in the website without the need of an app (actually I could never use any banking app as my telephone lacks Google Play).