You can add them by creating a unique, temporary UUIDs/links that they can use?
You know them from somewhere else, lets say I play a game and we decided to get into a voice chat. We could create a temporary, dynamically created voice chat that we can all join (much like Google Meet) where all of us are anons.
Then, if we really want to know each other, we can then share the UUIDs.
I understand why ANYONE can send an email to me (I can decide when/will to check them)
I don't understand why ANYONE can whisper to my ears (I cant decide since they are pushed to the top of the app)
Adding this level of friction to the process is not viable for a messaging platform whose bread and butter is connecting with friends.