In my case, I don't think most people who claim HN is social media are making a serious argument. They are taking two different things which have a few points of inter-comparison and using that as a basis to claim those two things are actually equivalent. This is done as a retort (eg, "you say you hate social media, but you're on social media _right now_") rather than in service of a larger argument.
You wrote a paragraph to say you don't "think" the others are right. What is the definition that made you think this?
I'm making it as a very serious argument.
HN is social, it has an algorithmic feed, people upvote and downvote your content, hell it has a social credit score. The idea that HN somehow isn't "social media" is hard to take serious. This is Reddit for a niche audience.
The main difference is that HN has a small and relatively high quality community, plus the traffic is low enough that it gets a fair amount of manual moderation. It's still social media and if there were enough people here, we'd eventually read stories of kids who offed themselves over downvotes. But we're thousands, not billions, so the law of large numbers doesn't apply.
If your FB feed or Youtube feed is garbage, spend some time curating it. HN is mostly curated for you, which appears to be creating unrealistic expectations of the broader world.