Hello, we have a server dedicated to running icinga2 with icinga2web client installed as a web GUI. In our ecosystem we monitor around 500 non-unique services distributed across around 100 hosts. Each time anyone reloads icinga after making edits to config files or updating templates many of services displayed in icinga2web will appear as Critical with their check status being in a state that usually appear as OK. What can we do correct this?
many of services displayed in icinga2web will appear as Critical with their check status being in a state that usually appear as OK.
Are these services in a CRITICAL state until they are checked for the first time after restarting Icinga2?
Can you share a screenshot as well as the output and the URLs provided to those plugins? Which versions of Icinga 2 and Icinga Web 2 are involved, anything in the logs?
Please kindly follow the instructions in the FAQ (upper menu) too.
dnsmichi Sorry, I made this post just before the weekend and I'm not concerned enough with this issue to check on it during work hours. I just tried to replicate the issue and I think it's related to adding new host .conf files to conf.d and then reloading because I couldn't force the problem to come back yet. Scanning through logs and grepping for error doesn't produce anything unexpected.
Wolfgang That might be it but it only seems to occur after I've added new host configuration files. It's difficult to force it to replicate because this instance is in production currently and people don't like seeing red but I'll look into what I can do.
Output of incinga2 --versionCode
- icinga2 - The Icinga 2 network monitoring daemon (version: r2.6.3-1)
- Copyright (c) 2012-2017 Icinga Development Team (https://www.icinga.com/)
- License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl2.html>
- This is free software: you are free to change and redistribute it.
- There is NO WARRANTY, to the extent permitted by law.
- Application information:
- Installation root: /usr
- Sysconf directory: /etc
- Run directory: /run
- Local state directory: /var
- Package data directory: /usr/share/icinga2
- State path: /var/lib/icinga2/icinga2.state
- Modified attributes path: /var/lib/icinga2/modified-attributes.conf
- Objects path: /var/cache/icinga2/icinga2.debug
- Vars path: /var/cache/icinga2/icinga2.vars
- PID path: /run/icinga2/icinga2.pid
- System information:
- Platform: SLES
- Platform version: 12-SP2
- Kernel: Linux
- Kernel version: 4.4.74-92.29-default
- Architecture: x86_64
- Build information:
- Compiler: GNU 4.8.5
- Build host: lamb28
Modules Loaded: setup 2.4.1 director 1.3.1 doc 2.4.1 monitoring 2.4.1
dnsmichi I just got it to reoccur and it does only appear to happen after adding a new host .conf file to /conf.d/hosts.d/. It's not actually any of the services that appear critical but instead it's every host coming up as critical even though the ping check that's used as the default hostalive check returns PING OK.
I'm still interested in a config snippet and a screenshot from your web interface.