I think that's conflating distinct concepts. Numbers aren't a physical property. Whether or not they're a fundamental concept is largely a philosophical question.
Meanwhile temperature (for example), while physical, is a statistical property of a macroscopic system. It isn't fundamental - rather it's an abstraction over a (very) large set - but it is nonetheless a quantification of physical characteristics. Whereas a number, for example 2 rocks, is not quantifying any physical property of the individual rocks themselves.
The existence of a rock is definitely a physical property, and thus is the number of rocks in a collection a physical property, at least as much as temperature is it for a gas.