I usually just move all the files to a new directory (/oldroot) and pivot_root -- any open files reference the new paths. Then install into the newly empty root directory of the filesystem, reboot and delete the /oldroot.
That sounds like the best way if keeping the filesystem is an option. In my case I wanted to also change filesystems and apply FDE, which is possible to do if the original filesystem supports online shrinking but many do not.
Don't you get any errors even if you race immediately to start pivot_root? pivot_root also won't modify all open file descriptors at once. Seems it's not fatal, but have you managed to do this over ssh and not be disconnected?