We have automatically generated readouts that run hourly and daily, show metrics between control and variant across a ton of core business metrics, and we can even opt into measurement of some super esoteric stuff.

Granted we have a whole team supporting just that experimentation platform, but it's pretty cool. They even expose an MCP server for us to have agents do experiment analysis for us using.