Elastalert + passive check

Hey Icinga Community :slight_smile:

I would like to share with you my current project and get your feedback about.

I’m pushing ~50k logs/sec into Log infrastructure. Right now, I’m using this awesome tools to monitor them: elastalert (https://github.com/Yelp/elastalert).

For example, I receive a SMS alert if an administrator logon occurred from my infrastructure.

My idea is to push this data to icinga2. So that my team can handle more easily alerts, acknowledge them (elastalert does not provide ‘recover’ action). “One alerting system to rule them all”!

To do that, I’m writing a new alerter for elastalert to push check_result into elastalert. This is the biggest part of the deal for me ^^
But I’m not sure about how to create the passive check on Icinga. I guess I should put ‘false’ on ‘enable_active_checks’. Right? Because I don’t care of the freshness here.

Thanks for reading :slight_smile:

1 Like

If you don’t care about freshness, then yes, enable_active_checks must be set to false.

Elastalert seems to be written in Python, so you could either use one of the icinga2 api libraries around, or go with requests and your own small handler for the process-check-result action :slight_smile:

I based my code on this project: https://github.com/fmnisme/python-icinga2api

The Elastalert Icinga Alerter was not so much hard to write :slight_smile:

I’ve disabled the active_checks, but I still see a message alert about a late on the last check from Icinga Web. Is it normal? It seems Icinga still detect a bad freshness even if enable_active_checks is false. Not really critical for me because the configuration is working.

I would say Elastalert + Icinga is a killer feature for our usage to detect any security breach and application crash :slight_smile:

Hmmm, I think you probably also need to disable passive checks too. Might be worth a try for the dashboard.

I’ve shared this post on twitter, and it gets love over there too :heart: