Grafana Template for internal icinga2 checks


#1

Hello,

I started to check my icinga2 cluster with icinga, cluster and the ido check. Now I write all the perfomance data to my influxdb.
Now I want to create grafs for that in grafan but I rember that someone build these grafs für grafana longe time ago.
Do someone have a hint for me where I can find the json code for these grafs?

Cheers, Mordecaine


(Carsten Köbke) #2

You will find the ido in my repository. I just made you one for the icinga check itself, for cluster you should try it by yourself :slight_smile:

itl-icinga-1530805409047.zip (3.3 KB)


(Carsten Köbke) #3

Fix some typos in the influxdb dashboard and created it also for graphite.

InfluxDB version:
itl-icinga-1530855515027.zip (3.4 KB)

Graphite version:
ITL-icinga-graphite-1530855363007.zip (3.2 KB)


(Blerim Sheqa) #4

Thank you for the dashboards @Carsten!
Would be nice if you would upload them on grafana.org as well. They even support Icinga as a collector for filtering: https://grafana.com/dashboards


(Carsten Köbke) #5

Will do that this evening.


#6

Hello Carsten,

thank you so much. That’s brilliant.

I have one more question. You said, that you still have enabled the ido check.
I tried it yesterday as well, but the result is not what I expect.

I still have three zones: master, checker and global-template. So I did the following:
I created a new service file for the master section:

apply Service "ido-mysql" {
  import "dci-1m"
  check_command = "ido"

  vars.ido_type = "IdoMysqlConnection"
  vars.ido_name = "ido-mysql" //the name defined in /etc/icinga2/features-enabled/ido-mysql.conf

  assign where match("my-masters-ma0*.example.com", host.name)
}

Now I did a syntax check (icinga2 daemon -C) which is going well. After that I did a reload and now the check is pending… since 24 Hours.

Im not able to see my fault.

There is one more question. I use the “ido” command, but I’m not able to find the Command Object in the icinga2 itl. So where I can find the “ido” command Object?

Cheers, Lasse


(Carsten Köbke) #7

You miss the command_endpoint :slight_smile: You need that, because the check must run on the node that has the ido connection configured. Just add this line ad it should work

  command_endpoint = host.name

The command is an internal icinga2 command, the documentation is here


#8

Hello Carsten,

I added the directive as you wished. But when I do an icinga2 daemon -C I get the following error message:

critical/config: Error: Validation failed for object 'my-masters-ma0*.example.com!ido-mysql' of type 'Service'; Attribute 'command_endpoint': Command endpoint must be in zone 'checker' or in a direct child zone thereof.

I added the service to the master zone. Is that wrong? But the checker servers didnt have the ido feature enabled, so they dont know the object command.

Cheers, Lasse


(Carsten Köbke) #9

Then your apply rule does match to much.
If it is only 1 server make it a normal service object, if more make the assign something that matches only the hosts with ido connections


#10

I think the service match. Because I get the error message twice and I have to master servers.
Further more the error message includes the servername (I accidentally added an asterisk when I changed the servername):

warning/Zone: The Zone object 'checker' has more than two endpoints. Due to a known issue this type of configuration is strongly discouraged and may cause Icinga to use excessive amounts of CPU time.
critical/config: Error: Validation failed for object 'my-master-ma01.example.com!ido-mysql' of type 'Service'; Attribute 'command_endpoint': Command endpoint must be in zone 'checker' or in a direct child zone thereof.
Location: in /etc/icinga2/zones.d/master/services.conf: 1:0-1:24
/etc/icinga2/zones.d/master/services.conf(1): apply Service "ido-mysql" {
                                              ^^^^^^^^^^^^^^^^^^^^^^^^^
/etc/icinga2/zones.d/master/services.conf(2):   import "dci-1m"
/etc/icinga2/zones.d/master/services.conf(3):

critical/config: Error: Validation failed for object 'my-master-ma02.example.com!ido-mysql' of type 'Service'; Attribute 'command_endpoint': Command endpoint must be in zone 'checker' or in a direct child zone thereof.
Location: in /etc/icinga2/zones.d/master/services.conf: 1:0-1:24
/etc/icinga2/zones.d/master/services.conf(1): apply Service "ido-mysql" {
                                              ^^^^^^^^^^^^^^^^^^^^^^^^^
/etc/icinga2/zones.d/master/services.conf(2):   import "dci-1m"
/etc/icinga2/zones.d/master/services.conf(3):

critical/config: 2 errors

(Michael Friedrich) #11

The command_endpoint is put onto the client, but the master doesn’t know about the relationship between zones. Guess the client zones are not configured on the master, as otherwise the config validation would not moan.


(Carsten Köbke) #12

@bobapple Posted both version :slight_smile: