> Imagine that a future AI system proposes a radically new design for a one terawatt nuclear fusion power plant

This is an unfortunate example to choose, being as it is entirely confounded by the canonical illustration of the https://en.wikipedia.org/wiki/Law_of_triviality.