> It's physically not possible at our current level of technology to make this "safer" due to the distances and energies involved.
That's not true at all.
It is entirely within current technical and fiscal means to launch a much more robust and powerful craft that is capable of goign to the moon and returning with lower velocity by sending it up in pieces with Falcon 9 (Heavy) and assembling it in LEO before launching to the moon.
This mission architecture is intrinsically compromised by social constraints in the form of pork barrel spending dsfunctional decision making process.
Given current levels of technology, this would require docking with a series of space tugs. Not impossible, but Blue Origin is the only organisation working on this at a meaningful scale.
There was also Nautilus-X which never made it beyond the concept stage.