The real innovation here, IMO, is theorem provers. Without theorem provers, the LLM brute-force-by-bouncing-off-the-guardrails approach just doesn't work to begin with and the whole thing spins off into hallucination and dodgy reasoning, just as we see in non-trivial AI software. Once you've got some magic oracle that gives you a binary yes/no answer to every attempt at a proof, your search strategy can be enormously simplified.
It's not really the provers that are the innovation. They existed before AI. The innovation is that the computer can write the proofs.
I also thought that, but I learned that most of OpenAI's results in their proof dump did not use theorem provers in the loop. They translated the papers to lean afterward.