> Why does it need to be? Would freedom of permanent movement still be something Europeans would vote for today?

Freedom of movement for labor is absolutely critical to counterbalance the freedom of movement that capital has, otherwise it leads to mass exploitation of labor and rising levels of inequality, which leads to, well, the French approach to the bourgeois problem.