Feel exactly the same way. I start going to shows in the late 90s - once in high school. All small venues.
I started going to more shows in college (mostly jam) and then even more as an adult with just a smidge more money.
Two shows stick out as particularly bad:
1. Dave Matthews Band - Fenway Park. There is no way to correct all of the oddities (and charm) of the place. The sound was terrible. I enjoyed the show anyways, but it was the worst sound quality I had seen for $90.
2. Phish - Fenway Park. Sound terrible too. From what I saw, the Phish show is folks listening to the music and folks doing whatever-else to the music. I enjoyed being with friends and people watching but nothing else. Luckily, scalped tickets were cheaper once the show started.
In contrast, went to many shows at Cambridge House of Blues, Boston's Paradise Rock Club, and many other similarly sized venues. Best sound, best experiences.
Lucky now to live near the Capitol Theatre in Port Chester NY - a true gem among all the other venues around.
The one great stadium experience I had was seeing Billy Joel on his River of Dreams tour. I had seats behind the stage about six rows back. Billy Joel did about half the show with his piano lifted up in place of the drums and Liberty Devitto moved to where the piano had been, so suddenly, most of the people had the behind the stage seats and I was in row six. But other stadium/outdoor amphiteatre shows I’ve been to (Genesis, Peter Gabriel, The Who) have been kind of underwhelming affairs.