Very unproblematic. Your travelling back will land you in a freshly branched universe with no way back to the one you came from, so no worries there.