This question may not be completely related to icinga2, but I am sure I am not the first one to have this doubts/need for this functionality, So I thought maybe someone could share his findinds/solutions.
I have made some tests with SNMPTT and SNMPTRAPD in order to receive snmp traps on icinga2, however I think we all agree icinga2 is not really traps focused. The system works and after receiving a trap you can modify a service on icinga2, send and email, etc. but what I cannot figure out is how to correlate incidents. I will try to give an example.
Let us supose that 3 ports on a switch go down (ports 1,2 and 10), that will generate 3 traps indicating the ports going down. After receiving the traps we could set a service or services to a critical state for example.
Then after 5 minutes ports 1 and 2 go up, and new traps are generated. After receiving this new traps we would ideally clear the critical status for ports 1 and 2, while keeping the status for port 10.
In this situation, I see two options.
1. Create one service per every switch port. The inconvenient is this would mean literally 1000+ services for our monitored network
2. Have some logic correlate the Up and Down traps.
I know we could potentially create the logic needed for this correlation, but I do not plan on reinventing the wheel, so is there any opensource software arround that does this correlation? I have taken a look at SEC and eventDB, but what does people usually use in this scenario?
Thanks for your time