This came across as a little odd and nerdy, but I'm actually really glad you shared your internal dialogue around this. It gives me more empathy for socially anxious (or just socially inexperienced) folks. Although the way you're starting out is kind of nerdy and overanalyzing, I'm sure these interactions will come naturally if you keep it up. Connecting with people is a very worthwhile effort and it's great that you're doing it.

In particular, the "rejection" will stop feeling awkward. I have random little one-or-two sentence exchanges with people several times per day, and usually it doesn't go beyond that, but I don't experience this as failure or rejection. I only engage further with the people who show (by words, body language, etc.) that they're genuinely interested in a conversation. For me, it's less than half.

The gym is an ok place, but not a great place, for what you're trying to do. Hiking clubs, running clubs, CrossFit gyms, rock-climbing gyms, and volunteer groups are all better options. The baseline level of socialization is very high in these places, whereas if you look around at a gym, most people have their headphones on, and are doing their own workout, so there's few natural opportunities to start a conversation.

Also, try to find people who are social and have lots of friends. If they like you they'll introduce you to their friends, which is a lot easier than starting cold. Don't be afraid to talk to women. Most of the people I know who are really good at connecting people are women.