logoalt Hacker News

thorian1828i03today at 3:04 AM1 replyview on HN

> The same thing with metric cardinality: by default, only report metrics in a fully aggregated manner. But do tell the monitoring system how they can potentially be broken up if needed (i.e., which labels to add).

How does the monitoring system have any of the context to add labels? That would only exist in application memory.

Grafana went the other way - your app exports all labels, and then you selectively aggregate on ingest: https://grafana.com/docs/grafana-cloud/observe-and-act/adapt...

> That may be prohibitively expensive in terms of CPU/network load.

In practice I've not experienced this even on quite high request rates. While it isn't free, exporting everything has been cheap enough that the real cost in dollars spent is basically marginal (it's _storing_ the data that's expensive)


Replies

EdSchoutentoday at 3:14 AM

> How does the monitoring system have any of the context to add labels? That would only exist in application memory.

Indeed. If you have a protocol that doesn’t allow exposing that kind of information, then that only lives in application memory. But my suggestion is that it’s exposed.