No graphs displayed for hosts in icingaweb2

icingaweb2

#1

Hi all,

I’m having a problem that my hosts in icingaweb2 don’t get graphs displayed from the graphite module, services are working fine.

When looking at the graphite access log on the graphite server i get lines link this, when refreshing the hosts detail page:

192.168.100.33 - - [06/Nov/2018:13:39:02 +0100] "GET /metrics/expand?query=icinga2.ST-CUCM1-OL.host.icmp.perfdata.%2A.value HTTP/1.1" 200 448 "-" "icingaweb2-module-graphite"

If I try to open the URL /metrics/expand?query=icinga2.ST-CUCM1-OL.host.icmp.perfdata.%2A.value in the browser I get a JSON string containing the different perfdata entries:

{"results": ["icinga2.ST-CUCM1-OL.host.icmp.perfdata.pl.value", 
"icinga2.ST-CUCM1-OL.host.icmp.perfdata.rta.value", 
"icinga2.ST-CUCM1-OL.host.icmp.perfdata.rtmax.value", 
"icinga2.ST-CUCM1-OL.host.icmp.perfdata.rtmin.value"]}

Checking the whisper files shows me, that there is existing data (I think):

# whisper-fetch --pretty /var/lib/graphite/whisper/icinga2/ST-CUCM1-OL/host/icmp/perfdata/rta/value.wsp
Tue Nov  6 15:52:00 2018        None
Tue Nov  6 15:53:00 2018        None
Tue Nov  6 15:54:00 2018        0.000420
Tue Nov  6 15:55:00 2018        None
Tue Nov  6 15:56:00 2018        None
Tue Nov  6 15:57:00 2018        None
Tue Nov  6 15:58:00 2018        None
Tue Nov  6 15:59:00 2018        0.000386
Tue Nov  6 16:00:00 2018        None
Tue Nov  6 16:01:00 2018        None
Tue Nov  6 16:02:00 2018        None
Tue Nov  6 16:03:00 2018        None
Tue Nov  6 16:04:00 2018        0.000412
Tue Nov  6 16:05:00 2018        None
Tue Nov  6 16:06:00 2018        None
Tue Nov  6 16:07:00 2018        None
Tue Nov  6 16:08:00 2018        None
Tue Nov  6 16:09:00 2018        0.000399
Tue Nov  6 16:10:00 2018        None
Tue Nov  6 16:11:00 2018        None
Tue Nov  6 16:12:00 2018        None
Tue Nov  6 16:13:00 2018        0.000448
Tue Nov  6 16:14:00 2018        None
Tue Nov  6 16:15:00 2018        None
Tue Nov  6 16:16:00 2018        None
Tue Nov  6 16:17:00 2018        None
Tue Nov  6 16:18:00 2018        0.000518
Tue Nov  6 16:19:00 2018        None
Tue Nov  6 16:20:00 2018        None
Tue Nov  6 16:21:00 2018        None
Tue Nov  6 16:22:00 2018        None
Tue Nov  6 16:23:00 2018        0.000678
Tue Nov  6 16:24:00 2018        None
Tue Nov  6 16:25:00 2018        None
Tue Nov  6 16:26:00 2018        None
Tue Nov  6 16:27:00 2018        None
Tue Nov  6 16:28:00 2018        0.000508
Tue Nov  6 16:29:00 2018        None
Tue Nov  6 16:30:00 2018        None
Tue Nov  6 16:31:00 2018        None
Tue Nov  6 16:32:00 2018        None
Tue Nov  6 16:33:00 2018        0.000589
Tue Nov  6 16:34:00 2018        None
Tue Nov  6 16:35:00 2018        None
Tue Nov  6 16:36:00 2018        None
Tue Nov  6 16:37:00 2018        None
Tue Nov  6 16:38:00 2018        0.000394
Tue Nov  6 16:39:00 2018        None
Tue Nov  6 16:40:00 2018        None

This is the corrensponding folder from the graphite-web interface, it resides in the icinga2 folder:
image
and data is shown:

The icingaweb2 details page of the host shows not even a hint of graphite, not even a “No data”.
And the Graphite menu shows “No graphs found” for every host.

Graphite Backend Config
image

Graphite Advanced Config
image

Anyone got an idea how to fix this?
I suspect that the GET call to graphite-web is not fully correct (%2A = * may be the problem?), but I couldn’t figure out how to fix it.

icinga2 v2.10.1
icingaweb2 v2.6.1
graphite v1.1.0

Thanks for the help,
Best regards


#2

I have reset my whisper files after editing my storage schema to

[icinga2_default]
pattern = ^icinga2\.
retentions = 5m:14d,10m:30d,30m:90d,60m:180d,1d:2y

The graphs are now more detailed in graphite-web
image

but still nothing in icingaweb2

here a two entries from the graphite-web acces log.
1st row from graphite-web, 2nd from icingaweb2

192.168.3.30 - - [09/Nov/2018:15:23:57 +0100] "GET /render/?width=586&height=308&_salt=1541773442.679&target=icinga2.FR-WLC.host.icmp.perfdata.rta.value HTTP/1.1" 200 38434 "http://st-grafana-rz:81/composer/?" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36 Vivaldi/2.1.1337.36"
192.168.100.33 - - [09/Nov/2018:15:24:09 +0100] "GET /metrics/expand?query=icinga2.FR-WLC.host.icmp.perfdata.%2A.value HTTP/1.1" 200 428 "-" "icingaweb2-module-graphite"

I think that .perfdata.%2A.value this is the issue, but I have now idea how/where to fix it.
The host name template fits otherwise, as the screenshot shows


#3

Problem finally solved (after hours :upside_down_face:)

The module comes with a set of pre-defined templates for some commands and default templates for hosts and services.

As I am using the icmp (and rarely) the http command to check my hosts there was a problem with the pre-defined templates. The graphite module matched the existing templates to the check commands icmp or http, but those templates are designed only for service checks.

So I copied the icmp.ini und response-*.ini and made a host equivalent for each of them (replaced “service” with “host” in the .ini file)

The graphite modules troubleshooting page gave me the final hint, to use &graph_debug=1 on the host page of icingaweb2

This showed that the graphite module tried to apply service templates and excluded the metrics to the host graphs (I think)