-
Notifications
You must be signed in to change notification settings - Fork 236
Description
Bug Report
What did you do?
Using operator with metrics. Implementation of Metrics (io.javaoperatorsdk.operator.monitoring.micrometer.MicrometerMetrics) is using
io.micrometer.prometheusmetrics.PrometheusMeterRegistry.
This produces a lot of warnings in logs:
Prometheus requires that all meters with the same name have the same set of tag keys. There is already an existing meter named 'operator_sdk_events_received' containing tag keys [action, event, group, kind, name, namespace, scope, version]. The meter you are attempting to register has keys [event, group, kind, name, namespace, scope, version].
It happens in the following method:
Line 169 in 6eb2486
| public void receivedEvent(Event event, Map<String, Object> metadata) { |
Mentioned implementation allows registration of the same metric but with different set of tags. This is not allowed by PrometheusMeterRegistry - see as prometheus treats this as separate time series see
What did you expect to see?
I expect to see all metrics without plenty of warnings in logs (Splunk).
Possible Solution
As operator uses 2 types of events - resource related event with action and non-resource related - I would expect to have the same set of tags, but in case of non-resource related maybe something general.
public void receivedEvent(Event event, Map<String, Object> metadata) {
if (event instanceof ResourceEvent) {
incrementCounter(
event.getRelatedCustomResourceID(),
EVENTS_RECEIVED,
metadata,
Tag.of(EVENT, event.getClass().getSimpleName()),
Tag.of(ACTION, ((ResourceEvent) event).getAction().toString()));
} else {
incrementCounter(
event.getRelatedCustomResourceID(),
EVENTS_RECEIVED,
metadata,
Tag.of(EVENT, event.getClass().getSimpleName()));
Tag.of(ACTION, "NA");
}
}