I'm always surprised to hear this, and I want to be clear that I'm not trying to be dismissive in my comment. However, I've not encountered issues while juggling dozens of Ruby projects since around 2011, despite seeing many people's complaints over the years. Ten years ago I was using rvm, and I saw people sharing their issues with it, and listing reasons why rbenv and chruby are better. So I tried those, and my resulting workflow felt basically the same once I got used to the differences.
At this point I've used rbenv, rvm, asdf, mise, and one other whose name isn't coming to mind. Not to mention docker containers, with or without any of those tools.
I don't mean to project any particular complaint onto you, and I'm curious what part of it is infuriating? Each of the version managers I've used has functioned as advertised, and I'm able to get back to work pretty smoothly.
My experience with the ruby ecosystem has been that if you get everything set up correctly all of the environment management tools have worked wonderfully. When you don't have everything set up correctly, they break in ways that is hard to understand for someone not intimately familiar with the ecosystem. It's something that's not at all a problem for someone using ruby as their primary language, and a major source of pain for dabblers and people who just want to run something written in ruby.