Mathematical concepts don't have to have an obviously physical analogue. I mean, you'd find it difficult to observe minus two apples and certainly tricky to observe i apples.

To my mind, maths is like a "what if?" puzzle and whether or not infinity makes sense in the physical world, there's still fun to be had by considering the consequences of it.

That also means that it can be interesting to consider limited number systems which don't have any concept of infinity.