Secrecy of correspondence still has exceptions. That's what is always lost in these discussions -- every right of every person is not absolute. Just because you have a right to personal property, doesn't mean you don't have to pay taxes or store nuclear material in your basement. That's the hard part.
But end to end encryption with forward secrecy at no cost to user makes your right to private communication absolute. It's a new thing and the balancers can't balance it against other rights of other people, so this happens.
The ends still have the decryption keys, so the result is the same as with a physical letter: you have to acquire the physical object holding the key material.
> But end to end encryption with forward secrecy at no cost to user makes your right to private communication absolute
As it should be. Governments should have to suck it up. If they want to know things about someone, they should have to actually assign police to follow them around. Not click a button and have the lives of everyone in the entire world revealed to them.