They were pointing your own faulty claim back at you, in the hope that you would reconsider it.

You should not be treating it as a standalone accusation, and it's not justified for you to get mad at them without getting just as mad at yourself.