icinga2 not sending notifications

This forum was archived to /woltlab and is now in read-only mode.
  • Hi All,

    we have icinga 2.4.7-1 installed in a cluster of two nodes. After upgrade notifications are not being sent out. Has anyone faced same issue before?

    icinga2 - The Icinga 2 network monitoring daemon (version: r2.4.7-1)

    Copyright (c) 2012-2016 Icinga Development Team (https://www.icinga.org/)
    License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl2.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.

    Application information:
    Installation root: /usr
    Sysconf directory: /etc
    Run directory: /run
    Local state directory: /var
    Package data directory: /usr/share/icinga2
    State path: /var/lib/icinga2/icinga2.state
    Modified attributes path: /var/lib/icinga2/modified-attributes.conf
    Objects path: /var/cache/icinga2/icinga2.debug
    Vars path: /var/cache/icinga2/icinga2.vars
    PID path: /run/icinga2/icinga2.pid

    System information:
    Platform: Ubuntu
    Platform version: 14.04.2 LTS, Trusty Tahr
    Kernel: Linux
    Kernel version: 3.13.0-53-generic
    Architecture: x86_64

    • From which version did you update ?
    • Does grep -i 'sending notification' /var/log/icinga2/icinga2.log return something helpfull ?
    • Are custom notifications comming through ?
    • Are notifications still enabled (icinga2 feature list) ?
    • Updated from (version: r2.4.3-1) to (version: r2.4.7-1)
    • Yes there's entry [2016-05-06 07:03:39 -0700] information/Notification: Sending notification; 'sending notification' in icinga2.log
    • Yes custom notifications coming through. Even notification for many hosts come through. Not sure for few hosts and services why it's not coming. Contacts are valid and cross verified them.
    • Yes notification is enabled.
  • OK, so the basic setup seems to work.
    If you do a icinga2 object list --type notification | less you can compare notifications that work with these that do not (if they exist at all).

    Double check properties like interval, period, states, templates, times, types, user_groups, users.

    I hope that helps to narrow it.

  • Yes I can see that in object --list notification.

    It's not changing number of attempts after which it should send notification. For eg. I have one service crtitical for long time. In that time it should have maxed out attempt many times and sent many notification. But it is not changing number of attempts it made.

  • It's not changing number of attempts after which it should send notification

    Here you seem to talk about the service, not the notification.
    So, compare a running to a non-running service on both master and satellite:

    icinga2 object list --type service | less

    So that you find out why the service object does not go to the hard state.

  • Have you moved the config from master repository down to conf.d at the satellite ?

    You need to manage that it exists on both machines.

    The post was edited 1 time, last by sru ().