Receiving email notifications when I'm not supposed to

  • Hey all,


    I'm running this very peculiar problem right now. I am receiving email notifications when I'm not supposed to, meaning that I have acknowledged "unknown" services that still send emails to me like it would if I had not acknowledged it already. Additionally, I have an agent installed on one of my Windows machines that says that it is not connected, even though on the web interface, it shows it's connected. See pictures below.


    Windows Agent showing not connected via email notification

    Sending notification for unknown port that I have already acknowledged

    Icinga2 Web Interface proving that I acknowledged this already a month ago and that the "last notification" was sent on that date (even though I just started receiving them again)


    Icinga2 web interface proving that the Windows agent is connected (yet, the email notification says otherwise)


    I've checked /var/log/icinga2/icinga2.log but didn't find anything of use. Where else should I check? It seems as if this problem started about 3/8/17 (today is 3/13/17). All I've done since then was install NagVis. Also, it seems to be that these emails send at 30 minute intervals (at xx:15 and xx:45) consistently. I have tried removing the acknowledgement and readding it, to no avail.


    Does anybody know what the problem could be?

  • Hi,


    you can try to set the interval attribut to 0 (default is 30 minutes). See Icinga Docs for further information about the custom attribut.


    But obviously the e-mails shouldn't be send if the service is aknowledged. Can you post your configured notifications?

  • I did not understand the issue at first glance. Actually you're mixing 2 issues here, so the screenshot with the unknown not connected check is totally unrelated to the acknowledgement issue right? Can you clear things up a little and we just focus on one problem, then the other?

  • mcktr


    I am not certain if putting up all my mail configurations would be useful because emails do send correctly, other than this problem. Did you want to see something specific?


    Currently, I have all types of notifications configured to be sent to me which includes the following:

    Problem, Acknowledgement, Recovery, Custom, FlappingStart, FlappingEnd, DowntimeStart, DowntimeEnd, DowntimeRemoved


    I did add in the interval = 0 that you suggested, but it didn't seem to do anything. The notifications are still sending.


    --------------------


    dnsmichi


    Sorry for not being too specific. I thought that they might be related because they are both issues that are falsely sending emails out.


    Here's my issues:


    1. I am receiving email notifications for services that I have acknowledged using the "acknowledge" button through the Icinga2 web interface. These services have been acknowledged for about a month or so (with no notifications being sent out), but just recently it started to send out notifications again even though they are still "acknowledged".


    From the pictures, you can see that on the web interface it shows that the last notification was apparently sent on February 7th, but from my email you can see that I am still receiving email notifications about the same acknowledged service currently.


    2. One of my windows agents shows that it is not connected via email, but it actually is connected (as confirmed by the web interface).


    --


    Here's a list of things that I've done recently (in order):

    1. Updated Icinga2 to v2.6.1

    2. Installed NagVis and Livestatus (but have disabled Livestatus and am using IDO instead)

    3. Switched from perl script for sending mail to Postfix


    After updating Icinga2, I did not experience any issues with any database problems. Same with installing NagVis. I just wanted to use Postfix because it seems like what most people are using for sending out email.


    Update:


    So, all of this actually doesn't matter. Turns out my Icinga server was cloned into another virtual machine as a CentOS 7 template. I thought I had turned off the Icinga2 service on this new cloned machine (service icinga2 stop), but I guess it was rebooted at some point (I was not working on this new VM) and since I had set up the template to start at boot ( systemctl enable icinga2) because it was my main Icinga2 machine, this meant that the Icinga2 service had rebooted, therefore causing it to basically run as another Icinga2 instance.


    Since the email notifications had been configured for this cloned machine as well, they were sending from that machine. And since the notifications look identical (because they are cloned), I was extremely confused as to how they were sending to me even though I had acknowledged them on the actual server but it turns out they were not actually acknowledged on the clone.


    This is a bit of an extremely rare case, but I guess it may happen to someone else at some point as well. Make sure you disable the icinga2 service if you happen to clone your Icinga2 server to function as something else!


    TLDR;

    I cloned my Icinga2 server and the clone began to send me emails after being rebooted.


    Thanks for the help anyways mcktr and dnsmichi . I am ashamed.

  • You could add a footer to your notifications which just calls hostname -f or similar. That way you'll always know the sender :) Might be a good addition to the existing scripts too.


    And don't be ashamed. I guess everyone has his/her moments staring at the screen and debugging the obvious ;)

  • Hi michi .

    I am newbie in linux .

    could you tell me the process of email alerts of icinga.

  • Hi michi .

    I am newbie in linux .

    could you tell me the process of email alerts of icinga.

    Hi, you should open a new thread and asking for help.

    Do not reply on a post that is resolved.

  • Please open a new thread with your question(s), if the documentation on notifications is not sufficient: https://docs.icinga.com/icinga…ring-basics#notifications (I suppose you did read it?)

  • Hi All,


    I have installed the ICINGA in my local server .i.e CentOS release 7 ... so now all is set but am unable to set mail notification someone kindly help top set the notifications....

    needfull.