Hi to everybody,
My name is Daniel, and I use Icinga2 to monitor my systems and my clients' systems. We have detected a problem two days ago and we have not been able to find an answer on the internet or on the forum. We have a master node with icingaweb2 and graphite to have historical performance data.
Two days ago, we detected a free space problem in the icinga master hard disk. We check the graphite information about the hard drive space used in the master node, and we watch a constant grown in the consumed space. In that moment, we send in the o.s. the folder that has the space, and we detected it was in /var/lib/graphite/whisper.
We analyze all data, and we detect that the folder "network statistics" of all our systems occupied a lot. We connect to graphite web interface, and we found the performance data of the network card was storing in a very bad format. Instead of having a graphic per physical network interface, there was a lot of different views, with the performance data in the category. We attach and image for optimal viewing. We have detected that problem in our clients that have updated the windows client to 2.4.4 or higher. In 2.4.3 is not present the bug.
We have updated the schema of graphite to the new schema incorporated in version 2.4, but the problem is still there.
What's your opinion?
We don't know what to do, apart of removing network statistics information from our systems...
Thanks a lot.