Apple does something very similar with Apple Private Cloud Compute. It's interesting cause their isolation argument is different. For instance, memory is not encrypted (so weaker protection against physical attacks), but they measure and guarantee integrity (and need to trust) all code running on the machine, not just inside the secure enclave.

Good question is how many lines of code do you need to trust at the end of the day between these different designs.