What you said comprises the exact error in logic that people make. Because we did this, this other thing is possible.

The EU was a certainty in a region that is hostile to change, wants big government, wants centralization, is suspicious of democracy, etc. Free trade would be a massive change, that is why it hasn't happened. The EU is basically the logical conclusion of European forms of governing.