I personally don't participate in it and don't really find it interesting at all. For the longest time it was fairly normal to bet among friends, but I do know some people who seem to obsess now over sports betting to what feel like unhealthy levels.
I really don't care and I'm definitely on your side where I think people should be allowed to do what they want in this regard.
The crossing of line for me though is that this is now being advertised on TV and practically everywhere and being normalized for children and teenagers.
Call me a cynic, but I think the real goal in the long term for these companies is to get people addicted to it and to normalize it from a young age while they're more impressionable and thats where I believe the true harm is.