They're only half right. The formalization of physics is called Hilbert's Sixth Problem[0] and there has been progress formalizing specific subdomains just fine for practical purposes.
But there's no unified theory, as quantum field theory and relativity don't get along and we still need a model for quantum gravity. So it's not possible to formalize the whole system because we genuinely don't know how the whole system works to form it's axioms.