Limit notifications in a certain period of time


#1

Hi,

we had a problem with a SMB share which affects at least 30 services, directly and indirectly, so we received 30 notifications within 10 minutes, which obviously had one cause. In our old monitoring we used a bash script to send SMS notifications and create Jira tickets with nagios commands depending on how many notifications were send in the previous 30 minutes to avoid spam, and we set the count to 0 every 30 minutes.
Is it possible to have a limitation for notifications in a certain period of time? or should I use a counter file as well?

How I understand the Restrict to nth to mth notification or Throttle periodic notifications in the conditions of a notification are generally applied and not for a specific period of time, correct me if I am wrong please.

The only solution I came up with, is to use a counter file as we used to do, or using Redis, which is an overkill at this point.

Thanks for your suggestions.


(Philipp Näther) #2

You could put all the depending services and the SMB service into a BI aggregation. Disable all notifications for the services so you only receive the notification of the BI aggregation every time one of the services goes warn/crit.


#3

Hi Philipp,

thanks for the suggestion, I need to take a deeper look in to it, and check if it will satisfy our need, hence we have ~ 1500 hosts with ~ 60 K services with cross dependencies. Which means identifying all the dependencies and implement the BI aggregation for all of them in a meaningful manner.

Thanks Philipp