I wrote an agent which uses event stream API to collect all events from icinga2
This is a long HTTP connection, icinga2 keeps responding events, and I caught all Exceptions, if there is any connection problem, it will retry to fetch events after 30 seconds.
However, I found sometimes, icinga2 stops responding events or responds few events without any Exceptions. I cannot find this unexpected situation except my monitoring system reports there is no or less
I watched /var/log/icinga2/icinga2.log, there is nothing valuable, and CPU & Memory usage look good.
So, is there any good idea to trouble shoot why icinga2 stops responding? Or is there any best practice for this API usage?