That understates what LLMs are capable of doing. They can and do build things outside of their training dataset. As long as you can get there by following existing concepts, they can do it.
Perhaps outside their training set in a strict sense of “this thing itself was not specifically in the training set” (although only perhaps - users often iteratively coax frontier models into achieving things just beyond their envelope, and those achievements could well be folded into the next model’s training set without you or I being aware of that). But even then, it is not directionally outside the training set: it steps outside not in the direction of the unknown, but in the direction of another part of its training set. An evocative metaphor would be dashing from cover to cover in the rain, because it lacks the umbrella or raincoat or simply the fortitude to tolerate getting wet that it would need to reach an uncovered destination.
Perhaps outside their training set in a strict sense of “this thing itself was not specifically in the training set” (although only perhaps - users often iteratively coax frontier models into achieving things just beyond their envelope, and those achievements could well be folded into the next model’s training set without you or I being aware of that). But even then, it is not directionally outside the training set: it steps outside not in the direction of the unknown, but in the direction of another part of its training set. An evocative metaphor would be dashing from cover to cover in the rain, because it lacks the umbrella or raincoat or simply the fortitude to tolerate getting wet that it would need to reach an uncovered destination.