I’m using Icinga+Grafana+Graphite in order to show my metrics within dashboards and I’m facing some problems about perfdata and host removing.
In my project, I’m manipulating (add/remove/edit) my hosts via API and the problem is when removing them. There’s like a hierarchy between the hosts, and the father Id (generated when I am adding it) is referenced in the son’s name. E.g. Father_$fatherId_…, Son_$fatherId_…
Its usage is that I want to perform some counting procedures when querying values in Grafana/Graphite, using the name/prefix/whatever to filter stuff.
The thing is, when I remove a host its host and services perfdata remains in the system. An example is that I have now 6 hosts in my system but there’s like 19 perfdatas registries (so it is counting 19, not 6), because the removed host perfdata is still on the database. Even if I add the “same” host its name probably will be different because it depends on its father id.
OLT = father
ONU = son
fathers: name1, name2
name1’s son: name3
name2’s son: name4
Adding name1 then name2: OLT_001_name1, OLT_002_name2, ONU_001_name3, ONU_002_name4
Adding name2 then name1: OLT_001_name2, OLT_002_name1, ONU_001_name4, ONU_002_name3
Perfdata: OLT_001_name1, OLT_002_name2, ONU_001_name3, ONU_002_name4, OLT_001_name2, OLT_002_name1, ONU_001_name4, ONU_002_name3
I looked into the folder /var/spool/icinga2/perfdata and the files are organized by timestamp, so it’s kind of impossible to know what to remove in there since everything is mixed, we have different metrics from differentes hosts and services inside each perfdata file.
What I need to know is that is there anyway which I can remove a specific host/service perdata? or something that can be done through Graphite? I did some research but coudn’t find anything useful to my case.