Proton says they care about security and privacy but at the same time makes it impossible to use your own keys or properly export the original emails from your inbox. I really can't take this suggestion seriously.
That's not a good argument. The easiest way to undermine security of everyone is to allow portability of keys features. Look for example at where Signal fails and for no benefit to a normal user.
I haven't tried myself but is this insufficiently proper? https://proton.me/support/proton-mail-export-tool