That's kind of how I feel about most pets.
I've thought about getting a pet turtle or tortoise [1] because they are my favorite animal, but I found out that in order for them to be happy and healthy they need a lot more room than I could easily fit in my house. Either a very large aquarium or a very large area for them to walk around depending on the species, neither of which I can easily have in my house.
And I think a lot of animals are like that. Ultimately a lot of these animals evolved in areas that really aren't that "confined" in any meaningful sense, and forcing confines seems kind of cruel.
[1] To be clear, ethically, not one of those shady endangered black market things that you can find.
We adopted three kittens that were found locked into a suitcase and thrown into the trash. Our house is in hills with coyotes so these cats would not survive for any length of time outside. They'd probably also be sent to the pound if we didn't adopt them. I feel bad for confining them in our house but I don't know if there would have been a better outcome for them.
Totally agree on more rare/exotic animals though - they shouldn't be subject to unnatural conditions like this.