To add on, while it doesn’t work with GenAI models as far as I know. AWS has a service around explainability around ML decisions

https://docs.aws.amazon.com/sagemaker/latest/dg/clarify-mode...