For the "canonical" physics explanation, see:

https://en.wikipedia.org/wiki/Past_hypothesis

You can happily run physics backwards and forwards, and there's no difference. But assuming that entropy was very low in the distant past is very useful and lets us talk about fascinating things, like what happened last tuesday (or 12 billion years ago)

If you just know the current state of the a system, there isn't really any difference between running time backwards and forwards. Of course, if you know exactly the state of every particle, field, etc, and the past hypothesis is true, running time one way will lead to lowering entropy, and the other way will lead to raising entropy. But if you're a mere mortal and just know the macrostate, either way you run time, the entropy of the system will increase.