I have so many problems with using Matrix with multiple devices and most of them are caused by its encryption. If there was a magic wand that would add encryption to a system without changing anything else there would be no reason not to use it, but the reality is that E2EE does place limits on the overall functionality of the system.

This is why Telegram and Discord are so popular. They are popular because they work well, and part of the reason they work well is not bothering with E2EE. For instance, when you join a group chat, the server can just send you the message history (if enabled) and there's no need to negotiate keys with every other participant. There no "joining...", there is no "message will appear shortly...", you just press the button and you're in.

I've never seen an E2EE group chat that could remain stable with more than a few hundred participants. Even Matrix gives up and just makes it unencrypted at that point.