I get your perspective and I will admit there are surely cases like that - I still wouldn't say it's a vast majority.
Though that raises a question. For example if you were a LGBT person in Deep South and abused over that, who is to blame? Every individual who did that (unfortunately, in the name of religion), or the Church culture who allowed for that to be norm?
And anger over church. I'm Polish, and I'm bi. The most well-known bishops were politically hand-in-hand with the previous ruling party. I would hear in public state media how I'm a part of "rainbow disease" that's infecting the country. Church would often buy land for 1 (!)% of the price. At which point can I say that it wasn't just a few bad apples, but the whole structure who allowed for things to be that way?