I've heard this argument many times, there's a grain of truth and I strongly agree RE 'arhythmic and atonal trash', but it strikes me as disingenuous.
Classical music was somewhat elitist even at the time, coexisting with popular/folk music traditions. The Romantic period definitely saw music and art break away for traditional forms and conservative mores, but it wasn't a free for all.
The people who enjoy classical music today are largely the same people who enjoyed it at the time. Educated and 'enlightened' upper/middle classes, who sought music celebrating their aspirations. Freedom from political oppression and aristocratic control. A world where people could succeed based on talent, hard work, intelligence, and propriety. Where the middle class would have the power to reshape government and social institutions to advance their interests rather than those of the nobility. A personal experience of the religious rather than state ordained. The music pivots to natural, romantic, mystical themes as these people lost faith that industrialization will deliver for them.
Compare modern popular music which deals with universal themes - love, loss, sex, money, partying. You can enjoy pop music and still feel that these themes are not important to you.
People don't fake enjoying classical music to convince you that they're smart and cultured. They just gravitate to music which represents their idea of smart and cultured. The same is true for a lot of art.