You are aware of software verification? The AI can prove (mathematically) that its code implements the spec.
That just takes you back to the debate about the code being the spec.
That just takes you back to the debate about the code being the spec.