Transformers can actually learn pretty difficult manipulations, even how to calculate difficult integrals, so I don't agree that they can only solve problems they've learned by heart.

The reason I believe it can be achieved in this time frame is that I believe that you can do much more with non-output tokens than is currently being done.