Standard SRE tasks are bad benchmarks.

First of all, familiarity with open telemetry apis is not knowledge, they are arbitrary constructs.

We are implying that conforming to a standard is the only way, the right way. I would challenge that.

Assuming models were good at this tasks, we could only conclude that this tasks were trivial AND sufficiently documented. Assuming they were good at this type of tasks (they can be trained to be good cheaply, we know that based on similar acquired capabilities) making a benchmark out of it would be less useful.

But I am sure nobody really cares and the author just had to SEO a little bit regardless of reality