Monitoring load balanced services

I would like to monitor load balanced services. In the documentation you can find examples
where a service depends on another service or host. But in this case the services are on the “same level”.
Lets assume I have 5 webservices running on Port 80. Now I can monitor the cluster IP so see if the
service is running at all. But I would like to know the status of the five real servers behind the load balancer.
I don’t like the idea of just monitor all 5 web services on all 5 servers independently.
If one or two services are down I would like to receive only a warning because there are 3 still left to
do the job. But if 3 ore more are not reachable if would like to receive an critical.

Sounds like you want to have a look at the “Business Process” Module for icingaweb2.

1 Like

This could be also done by a runtime check in icinga2

A example is here:
https://www.icinga.com/docs/icinga2/latest/doc/08-advanced-topics/#access-object-attributes-at-runtime-cluster-check

3 Likes

Thanks! That’s what I was looking for :slight_smile: Almost! Because this is a host check the status can not be ‘Warning’. Is it possible to use a runtime check on a service as well?

Sure it is.

Here a example:

  vars.check_services = [ "someservice1", "someservice2" ]

  vars.dummy_state = {{
    var up_count = 0
    var down_count = 0
    var check_services = macro("$check_services$")
    for (service in check_services) {
    if (get_service("somehost",service).state > 0) {
        down_count += 1
      } else {
        up_count += 1
      }
    }

    total_count = down_count + up_count

    if (total_count != down_count) {
      return 0 //same up as down -> UP
    } else {
      return 2 //something is broken
    }
  }}

  vars.dummy_text = {{
    var output = "sometext:\n"
    return output
  }}

This can be used in a service.

1 Like

Thank you :slight_smile:

Is it possible to set this up (custom variables as functions) using director? I have a sync rule in place to import hosts and would like to add this to it, although I do not see a way to use function in director.