Repeat_key does not behave as expected

icinga2

(Yaron Idan) #1

I’m trying to use the prometheus nagios plugin in our Icinga instance and I’m getting some strange results.
I’ve configured the check_command for using the plugin like so -

object CheckCommand "check_prometheus_new"  {
  import "plugin-check-command"

  command = [ PluginDir + "/check_prometheus_new/check_prometheus_metric.sh", ]
  arguments = {
    "-H" = {
      value = "$prometheus_endpoint$"
      required = true
    }
    "-C" = {
      value = "['--cacert', '<redacted>', '--cert', '<redacted>', '--key', <redacted>']"
      repeat_key = true
    }
    "-q" = {
      required = true
      order = 2
      value = "$query$"
    }
    "-w" = {
      required = true
      order = 3
      value = "$warning_threshold$"
    }
    "-c" = {
      required = true
      order = 4
      value = "$critical_threshold$"
    }
    "-n" = {
      required = true
      order = 4
      value = "$name$"
    }
    "-m" = {
      required = true
      order = 4
      value = "$method$"
    }
    "-O" = {
      required = true
      order = 4
      set_if = "$ignore_nan$"
    }
  }
  vars.prometheus_endpoint = "<redacted>"
  vars.max_check_attempts = 5
  vars.check_interval = 60
  vars.retry_interval = 30
}

Now I expect Icinga will execute the check like so -
('/usr/lib/nagios/plugins/check_prometheus_new/check_prometheus_metric.sh' '-C' '--cacert' -C '<redacted>' -C '--cert' -C '<redacted>' -C '--key' -C '<redacted>' '-H' '<redacted>' '-q' 'scalar(min(up{source_prometheus=''}))' '-w' '100' '-c' '200' '-m' 'gt' '-n' 'experiment_prometheus'
But when I check the debug.log, I see Icinga ran the check this way instead -
('/usr/lib/nagios/plugins/check_prometheus_new/check_prometheus_metric.sh' '-C' '['--cacert', '<redacted>', '--cert', '<redacted>', '--key', '<redacted>']' '-H' '<redacted>' '-q' 'scalar(min(up{source_prometheus=''}))' '-w' '100' '-c' '200' '-m' 'gt' '-n' 'experiment_prometheus'

Shouldn’t the repeat_key option split the list and add the key in front of every option? That’s the impression I got from reading the docs. Have I configured something wrong?


(Nicolai) #2

Your value isn’t an array.

Nicolai


(Yaron Idan) #3

Hey, thanks for the swift response.
What do you mean it isn’t an array? because it’s quoted?
I’ve configured it according to the icinga2 documentation, which made me feel it’s gonna work.
I’ve also tried removing the quote to make it an actual array, and got the following error -

[2019-01-03 14:49:30 +0000] critical/config: Error: syntax error, unexpected $undefined, expecting ']'
Location: in /etc/icinga2/conf.d/objects/checkcommand/check_prometheus_new.conf: 13:16-13:16
/etc/icinga2/conf.d/objects/checkcommand/check_prometheus_new.conf(11):     }
/etc/icinga2/conf.d/objects/checkcommand/check_prometheus_new.conf(12):     "-C" = {
/etc/icinga2/conf.d/objects/checkcommand/check_prometheus_new.conf(13):       value = ['--cacert', '<redacted>', '--cert', '<redacted>', '--key', '<redacted>']
                                                                                       ^
/etc/icinga2/conf.d/objects/checkcommand/check_prometheus_new.conf(14):       repeat_key = true
/etc/icinga2/conf.d/objects/checkcommand/check_prometheus_new.conf(15):     }

Also tried converting the quotes surrounding the different values with double quotes and then got this error -
[2019-01-03 14:50:44 +0000] critical/config: Error: Validation failed for object 'check_prometheus_new' of type 'CheckCommand'; Attribute 'arguments' -> '-C' -> 'value': Invalid type.

Any ideas?


#4

If I’m not mistaken the solution would be to define the input filed for the “-C” parameter as an array. And then add all your values as entries for that array.

Example:
image

image

As you are not using the Director (and I only use it :D) I will try to find the correspondig config snippets :smiley:

Example of the arguments as array inputs for the field

vars.nrpe_arguments = [ "Daily-Backup", "1" ]

I think the command definition is just missing the field name. here is a different example from my config.

"--exclude" = {
            repeat_key = true
            required = false
            value = "$netapp_cdot_excludes$"
        }

(Nicolai) #5

Should work if you move the values of the command definition:

object CheckCommand "check_prometheus_new"  {
  // [...]
    "-C" = {
      value = "$extra_config$"
      repeat_key = true
    }

 // [...]

  vars.extra_config = [ "--cacert", "<redacted>", "--cert", "<redacted>", "--key", "<redacted>" ]
}


(Yaron Idan) #6

Thanks for your response but since I’m not using the director this is not a viable option.
We’re updating Icinga using the icinga puppet module so I can work with either puppet syntax or the raw icinga resource created by the director. Anyway you can supply me with the underlying resource created by the director when choosing an array datatype?


(Yaron Idan) #7

That totally did the trick, Thanks!


(Yaron Idan) #8

Thanks! That is indeed the correct answer, as @nicolaiB suggested.
Appreciate the help.