It's indeed thought to be really hard to factor multiples of two massive primes, but we don't know that for certain. See https://en.wikipedia.org/wiki/Integer_factorization -- "Unsolved problem in computer science: Can integer factorization be solved in polynomial time on a classical computer?". In fact, we don't even have a proof that this is more complex than multiplying the component primes.

And of course, many cryptosystems are reliant on primitives with much less algebraic structure.

But to be clear, LLMs would presumably break these cryptosystems by building new algorithms and writing code to break them, not by "intuiting" their way to a specific private key.