Matrix/element is the best bet atm. Had a solid 40m users compared to Discord's 140m at some point, which is good proof they can scale up compared to other. Hardest part is convincing your groups to switch from Discord. But you can bridge the two in Matrix and talk between them which might help ease the transition.
Teamspeak has been drama-free at least and spared from bigtech bullshit, but simple things like changing your username havent been working for like 6 years now, it is in rough but ok shape if you want a drop-in replacement. You can freely make group chats and categorize them by right clicking them to create Discord-style chatrooms or voice chat rooms. Otherwise you must pay to set up a Teamspeak community for your friends.
Matrix is awful. I don't know if that's still the case but when I tried it a year ago it took me more than 30 minutes to get it working. In discord, you sign up with email (very low friction), click the + button and boom. You've got a private server you friends can join by clicking a link. It Just Works and until an alternative has this level of convenience Discord is not going away.
This is to say nothing of the insanity of public, effectively unmoderated rooms on Matrix.