That’s fair. You can decline to participate in casual conversations and be annoyed.
Most people don’t mind someone initiating a casual conversation in a non threatening manner. Most will enjoy it, at least sometimes.
I’m happy for the author here, especially that he was able to shrug off these awkward interactions and move on.