...it does introduce certain caveats that, in my experience, are not frequently addressed in discussions about these systems.

What are those?

As far as I can tell, people don't deploy verified or high assurance systems without testing or without online monitoring and independent fail safes.