Why Icinga2 does not send cpu usage metrics to influxDB sometimes?


(Alireza ) #1

We send data from Icinga2 to influxDB (another remote server) in order to visualize data in Grafana. Until recently we didn’t experience a problem like jittered graphs on Grafana. It seems that there is null values in some periods for our server’s CPU usage (all of them at once).

We also have other metrics like RAM, Disk, Network Bandwidth but none of them has null values like CPU. We don’t have data for 6 minutes sometime.

Why this happens? How can I solve this problem?


(Michael Friedrich) #2

Either the check results are no returned, or it contains invalid performance data metrics. Could also be the case that metrics are not sent in the proper interval and InfluxDB throws them away based on some retention rules. Debug logs from both, Icinga 2 and InfluxDB will shed some light in here.

(Alireza ) #3

I could not understand what do you exactly mean by invalid performance data metrics? This is CPU usage metric which its data is generated by Icinga’s plugin.

InfluxDB retention policy, wipes out old data! It cannot purge data from the middle of the time.

Good to note that after restarting Icinga2 service, there is no more sparse data for CPU usage, but there are few for storage metrics :slight_smile:

Thank you for your time.

(Michael Friedrich) #4

Performance metrics follow a specified format. If that does not match for one data point, you won’t get a value. This could be the case for the plugin e.g. running into a timeout, and as such, not providing a) a value, just null b) an invalid format.

Still, this is a blind guess and logs will help here to analyse further.