In various sectors, you need to be able to explain why you/your-system did what it did. Exchange Act Rule 15c3-5 is probably the most relevant in financial circles:
https://www.ecfr.gov/current/title-17/chapter-II/part-240/su...
Note: I am neither a lawyer nor in financial circles, but I do have an interest in the effects of market design and regulation as we get into a more deeply automated space.
To add on, while it doesn’t work with GenAI models as far as I know. AWS has a service around explainability around ML decisions
https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-mode...