My biggest turnoff has been the fact that you don't own your own data/account and are beholden to whichever dictator(s) run the instance you started out on. You can migrate, but that entire process is just convoluted. I should be able to create an account with my own keys and use them anywhere. Servers can choose to use and share allowlists or blocklists. Each instance being its own little world kills discovery and adds a ton of friction.
And instances seem to be pretty heavy on resources. Reminds me of why Matrix never really took off, running a Matrix server is just too difficult and time-consuming for what you get out of it.
I know proponents of Mastodon will point out that you can work around these warts, but I don't want to. I don't think the model is suited for me.
I’m not 100% sure but I think you essentially described how Nostr works in your first paragraph.