> configure the benchmark to run for a long duration in a steady state: e.g., hours.

Benchmarking as a developer activity isn't going to cut it long term, that is at best a proposal to change.

If you do an active benchmark, which is basically running a workload when the SuT is tied up to a "data polygraph" of sorts, then you tend to accidentally build production observability tooling in the process.

The best kind of "benchmark" comes from a fleet in action, rather than something which was run in isolation. But those tend to be entirely a 1 of 1 test scenario rather than a real replicable result.

Most of my past performance work revolves around caches, those are exceedingly sensitive to workload patterns and what worked in May will fail in October.

With the whole cloud thing, the "workload pattern" might be someone else's.

My team put together a blog post last week which concluded that our coding models work better in PST afternoon rather than the mornings.