This is a fantastic article with lots of interesting questions raised between the halting problem and applications of AI.

The MRI example is very interesting - add a bit of noise and the crisp results fall apart. But we know we can carve out some determinism by orchestrating around LLM (validation loops) but they are only as good as the coverage of validation and the fidelity of expectations.

Which raises a fundamental question: where is the line between an effective harness and a broken one?

To put it in halting problem terms, if we have an outside program that checks the internal program for halting, we know we will capture some number of previously incatchable halting problems. How can we quantify this? We know there is a class of halts we can catch and prevent, but there will always be more complex ones that escape us.

In the real world, knowing the task is impossible doesn't impact the value of reducing the problem. More importantly, knowing which types of halting problems can be fully captured and which ones are intractable. Agent orchestration could directly benefit from layering a framework over this fuzzy space of knowableness.