Carbon Problem using Icinga2web-Graphite-Module

This forum was archived to /woltlab and is now in read-only mode.
  • So I installed the Icinga2 Graphite Module as per the following description:

    https://www.linuxfrickeln.de/v…en-in-der-icingaweb2-gui/

    The Graphite graphs are available within Icingaweb2. So I suppose it works.

    I must add that the part about installing Graphite-Web was already done before.

    So it was mainly about getting the Icingaweb2_Graphite-Module to work.

    But there is one thing in this documentation that does not work for me.


    In the instructions there’s the passage about restarting carbon-cache:

    Code
    1. systemctl restart carbon-cache.service

    When I try to use this command I get the following error message:

    Next there is the whisper fetch command. In the doc the path is :

    Code
    1. whisper-fetch --pretty /var/lib/graphite/whisper/icinga/hostname/rta.wsp

    But in my configuration it looks like this:

    Code
    1. whisper-fetch --pretty /var/lib/graphite/whisper/icinga2/hostname/host/hostalive/perfdata/rta/value.wsp

    It reads that the output should look something like this:

    Code
    1. Wed Oct 25 13:47:00 2017 0.000024
    2. Wed Oct 25 13:48:00 2017 0.000023
    3. Wed Oct 25 13:49:00 2017 0.000022
    4. Wed Oct 25 13:50:00 2017 0.000025
    5. Wed Oct 25 13:51:00 2017 0.000024
    6. Wed Oct 25 13:52:00 2017 0.000022
    7. Wed Oct 25 13:53:00 2017 0.000023
    8. Wed Oct 25 13:54:00 2017 0.000022
    9. Wed Oct 25 13:55:00 2017 0.000023

    But the output in my system looks like this:

    Code
    1. Thu Oct 26 10:42:00 2017 None
    2. Thu Oct 26 10:43:00 2017 None
    3. Thu Oct 26 10:44:00 2017 None
    4. Thu Oct 26 10:45:00 2017 None
    5. Thu Oct 26 10:46:00 2017 None
    6. Thu Oct 26 10:47:00 2017 None
    7. Thu Oct 26 10:48:00 2017 None
    8. Thu Oct 26 10:49:00 2017 None

    I believe this is because the carbon-cache cannot be restarted (see error above).

    In the carbon docs (http://graphite.readthedocs.io…bon.html?highlight=carbon) there’s a command to start carbon:


    Code
    1. /opt/graphite/bin/carbon-cache.py start

    But my /opt/ folder is completely empty.

    Instead there is a file in the /usr/bin/ folder named “carbon-cache” but the carbon docs refer to a file called “carbon-cache.py” to start carbon. There is no such file on my system.

    Can anyone help?

    dmf_hd

  • Excerpt from FAQ

    Quote

    Things you should always include

    • Which documentation source did you use (URL, short quote).
    • Distribution name and its version (/etc/os-release or /etc/*-release or /etc/*-version) of any involved host.
    • Software versions (<applicationname> --version or via web interface - About).
    • Installation method (packages, or source, and if Git, git show -1).
  • Things you should always include


    • Which documentation source did you use (URL, short quote).
    • Distribution name and its version (/etc/os-release or /etc/*-release or /etc/*-version) of any involved host.
    • Software versions (<applicationname> --version or via web interface - About).
    • Installation method (packages, or source, and if Git, git show -1).

    Pointer 1: is pretty much covered in my original post.


    Pointer 2: Debian Stretch


    Pointer 3: LAMP Server, PHPmyadmin, Icinga2, Icingaweb2, Graphite


    Pointer 4: mostly packages, Graphite from Github (see link in original post).

  • I am surprised that no one seems to want to help. I am pretty sure there's people with enough insight to graphite to be able to look into the issue according to the info and coding I supplied. I have been checking everywhere to find hints or even a solution on the net without much luck. So unless somebody can tell me what I have done wrong to not have a carbon-cache.py file and hence not being able to restart carbon I am pretty much stuck here.


    Thank you.

  • Since the log extract of your systemctl status command suggests an error regarding the config / the binary, please provide some more logs of the daemon. Adding your carbon-cache config could also help.


    And please note: In most cases getting no answer after some days doesn't means that noone wants to help. Everyone in this forum is doing his best and on on a voluntary basis.

  • I think can't help You with your problem here, but if you just started with graphite, i would switch to influxdb instead of graphite. Its a lot easier to install/configure in my opinion. On top the Database structure has the huge advantage that you don't need to preconfigure how often a value is coming in, so its easier to work with influxdb on the long term and you not have to fight with "Null" Values.


    I used graphite longer than influxdb, but i will never go back. influxdb had me after one day.


    Only one hint: I had problems with incompatible whisper things in python. Can't remember the exact solution, but from my old docs for Debian i had this snipped in my docs:

    Code
    1. apt-get install python-gdbm python-pip python-cairo python-django python-django-tagging python-dev
    2. apt-get install libapache2-mod-wsgi python-twisted python-memcache python-pysqlite2 python-simplejson
    3. pip install whisper
    4. pip install carbon
    5. pip install pytz

    But i remember messing around with other things to get it to work.

  • I am surprised that no one seems to want to help.

    Quote

    Software versions (<applicationname> --version or via web interface - About).


    Pointer 3: LAMP Server, PHPmyadmin, Icinga2, Icingaweb2, Graphit

    Looking at your efforts of answering simple questions in a suitable way some people able to contribute to the solution probably expect more helpful information from your side first.


    Since the posting of the instructions you used the software (OS, Icinga2, Graphite, ...) was enhanced so paths and several other aspects might have changed meanwhile. Some versions might have had bugs which were resolved in later releases so naming the products isn't sufficient.

    To use a real life example: Suppose the headlight of your car was damaged. Standing at the spare parts counter it doesn't help to tell the model but in most cases you need the model year or sometimes even the month to get the right part.

  • Since the log extract of your systemctl status command suggests an error regarding the config / the binary, please provide some more logs of the daemon. Adding your carbon-cache config could also help.


    And please note: In most cases getting no answer after some days doesn't means that noone wants to help. Everyone in this forum is doing his best and on on a voluntary basis.

    I know that most people here are doing this on a voluntary basis which I am greatful for. But not getting any response in over a week makes you initially think you are being ignored. I will see if I can dig up the additional coding.

  • Looking at your efforts of answering simple questions in a suitable way some people able to contribute to the solution probably expect more helpful information from your side first.


    Since the posting of the instructions you used the software (OS, Icinga2, Graphite, ...) was enhanced so paths and several other aspects might have changed meanwhile. Some versions might have had bugs which were resolved in later releases so naming the products isn't sufficient.

    To use a real life example: Suppose the headlight of your car was damaged. Standing at the spare parts counter it doesn't help to tell the model but in most cases you need the model year or sometimes even the month to get the right part.

    I see your point but I supplied the info to the best of knowledge. This is a work in progress for me between all the other stuff I am responsible for here at work. So, for instance, it's been weeks since I configured the LAMP server and Icinga2, Icingaweb2, Director and such. I don't make notes so I only know vaguely how I went along.

  • Since the posting of the instructions you used the software (OS, Icinga2, Graphite, ...) was enhanced so paths and several other aspects might have changed meanwhile. Some versions might have had bugs which were resolved in later releases so naming the products isn't sufficient.

    Ok, in this case, can you or someone else please refer a recent installation guide for Debian Stretch and instructions on how to completely delete all components of graphite?

  • https://www.linuxfrickeln.de/v…en-in-der-icingaweb2-gui/ was written in 2015 and is most likely using the old legacy format which was deprecated in 2.4. That explains the different whisper structure.


    You can find the current specification inside the docs: https://www.icinga.com/docs/ic…phite-carbon-cache-writer


    Unfortunately it is not very obvious from your initial post how you've actually installed Graphite with Carbon-Cache, Whispher and Graphite-Web. One could guess from the error log that you'll now have a separate Python installation in /usr/local/lib/python2.7 with different Python libraries installed compared to what the system's Python provides.


    https://graphite.readthedocs.io/en/latest/install.html suggests various dependencies. I guess Debian Stretch doesn't provide graphite-web for some reason, so you'll really have to stick with "pip install". Still, this makes it a pain to upgrade and maintain. History tells that Graphite and Debian don't play nice together when using packages, graphite-web always is missing.


    On the other hand - if you are not bound to Debian Stretch, consider looking into Ubuntu 16.04 LTS. A quick Google leads to this thread for example :https://linoxide.com/ubuntu-ho…phite-statsd-ubuntu-16-0:

  • Hi, thanks for the info. I understand that the installation instruction is rather outdated. A colleague I was talking to also notified me of a possible mistake I made during the install.


    To be honest I do not recall the exact procedures I used installing graphite and its dependecies. I went through varios documents I found and possibly mixed the procedures which did not really support my task of getting the thing to work.


    In general I got graphite-web to work in Icinga2. The graphite graphs are visible in graphite-web and also within Icingaweb2 but do not show any values. Somehow the forwarding of the fetched data must be obstructed.


    I am pretty much bound to Debian Stretch. It is the standard installation on our linux servers.


    I have set up a new LAMP server and will start a fresh install of Icinga2, Icingaweb2 and Director using the icinga2 documentation. After that I will install graphite and graphite-web as it is described here: https://graphiteapp.org/


    Concerning python my LAMP server already has python,python2.7, python3 and python3.5 folders inside the /etc/ folder. I will see how everything pans out on this try.

  • Then try to extract the systemd error message from the syslog, to see which stacktrace it exactly has. This allows for better googling the error message. Probably it has something to do with Twisted (a dependency from Graphite Carbon) and a version mismatch. I remember that I had to pin Twisted to a specific version with Graphite 0.9.12 (Puppet-Code inside the Vagrant-Boxes: https://github.com/Icinga/icin…manifests/default.pp#L547)

  • Then try to extract the systemd error message from the syslog, to see which stacktrace it exactly has. This allows for better googling the error message. Probably it has something to do with Twisted (a dependency from Graphite Carbon) and a version mismatch. I remember that I had to pin Twisted to a specific version with Graphite 0.9.12 (Puppet-Code inside the Vagrant-Boxes: https://github.com/Icinga/icin…manifests/default.pp#L547)

    You are talking about the "old" server, right? I am looking to go ahead with the new installation and (given it works) compare the 2 against each other to find the error. Nevertheless I will check the syslog.

  • The one which is mentioned in the initial post. Your last post didn't sound like you would immediately start from scratch. Although I think it is better to start fresh and document all the steps involved. This allows for later troubleshooting and you can also share them here in case of questions. Think of it this way - if one really has the time to reproduce the problem, clear cut installation instructions help save time.