You're confusing "contains information" with "has produced a result."

A proof being latent in an LLM is no more significant than a proof being latent in a book, a theorem prover, or the axioms themselves. Einstein's papers were latent in the genetic code of his parents and the environment of his time. That doesn't mean general relativity was "already done" before Einstein was born.

By your logic, no computation has ever accomplished anything because the output was always implicit in the inputs.

The entire purpose of computation is extracting information from representations where it's difficult to see into representations where it's easy to see.

So no, this isn't a problem with the original reasoning. It's a problem with yours.