We understand "time" in the context that we must: to subsist in order to procreate. The extent it exists outside of our own perception as we imagine it does, is debatable. Ultimately, time is our sensory interpretation of the world around us to facilitate our survival and thus, we may never make sense of it outside the constraints it exists within.
We know empirically that time only flows in one direction, it can’t be described as just a perception. You’d have had at least the tiniest of evidence that time sometimes flows backwards.