We have a monitor called device-events which looks for a set of eventid in logs tied to a device. We categorize the eventid as “set-eventid” and “clear-eventid”
For e.g. lets pick devicename as localhost-a and has 4 eventid where event id [1,2] are set event id and [3,4] are clear event id and the monitor will send a warning alert if it saw any of the set-eventid in last 10 mins and reset it as OK if we see any of clear-eventid. the plugin is set to run once every 5 mins. When monitor check runs it looks for all event id reported in last 10 mins range from elasticsearch index
We have a scenario where let’s say at 9AM the device has logged event id 1 and the monitor check runs at 9:01 AM
the icinga monitor output looks something like
“localhost-a has reported event id 1.” and we set the status as WARNING and send a notification for this alert
Now during the next poll we see event id 2 reported at 9:04 AM the icinga monitor output looks like
"localhost-a has reported event id 1.localhost-a has reported event id 2. " . But the status is still WARNING since it’s a set-eventid but there was a new event id 2 present in icinga service output which never gets reported. Is there a capability in icinga where we can know if the output of monitor has changed even though the status hasn’t changed and still send out a notification.
icinga2 - The Icinga 2 network monitoring daemon (version: r2.8.2-1) Copyright (c) 2012-2017 Icinga Development Team (https://www.icinga.com/) License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl2.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Application information: Installation root: /usr Sysconf directory: /etc Run directory: /run Local state directory: /var Package data directory: /usr/share/icinga2 State path: /var/lib/icinga2/icinga2.state Modified attributes path: /var/lib/icinga2/modified-attributes.conf Objects path: /var/cache/icinga2/icinga2.debug Vars path: /var/cache/icinga2/icinga2.vars PID path: /run/icinga2/icinga2.pid System information: Platform: Debian GNU/Linux Platform version: 9 (stretch) Kernel: Linux Kernel version: 4.9.0-9-amd64 Architecture: x86_64 Build information: Compiler: GNU 6.3.0 Build host: 022328c363ac```