I'm having trouble with my logic here, I've got a remote monitoring machine (It's behind firewall/private customer network) that creates an SSH tunnel back to our Icinga2 monitoring machine and it submits the remote checks using NSCA to submit the checks.
However, I'm struggling to get the check attempts/alerting right. The remote icinga install has the service defined as:
So it checks every 11 minutes and will check twice before it shows the service as HARD DOWN.
However, because it's using obsess_over_hosts (and services) it uses the command to submit these using NSCA.
It seems to send these regardless of the max_check_attempts (I'm guessing it's sending SOFT states as well as hard) but when it arrives at the NSCA/Icinga2 install these checks come out and alert immediately.
SO onto my question, on my icinga2 install (that receives the NSCA checks) should I have the service setup with
max_check_attempts = 1
So that it receives the NSCA alert and triggers an alert straight away, or should it be set the same so:
max_check_attempts = 2
(The latter of these when set to 2 never seems to alert as after the first send from NSCA it doesn't send another/repeat)
Hope that makes sense!