Posts by sru

Please read about the latest changes and upgrade.
Please note that your newly registered account needs to be manually activated by an admin. Just be patient, you cannot post to any forum before this happens. This is thanks to the unfortunate event of spammers recently. More details here.


    the grafana module uses the grapher hook.

    It presents rendered images of grafana panels, and that works great.

    Now i modified that module to present an iframe to the panel instead of its rendered image.

    I feel it would be a great improvement of the module to have the interactive grafana features at that position.

    But that iframe will get a refresh trigger every 10 seconds - resulting in a "curtain" effect because the panel often needs a second to show up.

    Any idea how to put the iframe out of the refresh interval ?

    You create services for hosts that have set.

    In the Service you set command_endpoint = host.vars.client_endpoint but vars.client_endpoint is not set in the shown host object.

    In the Check_Command you use vars.raid_env, but where does that come from ?

    According to the docs, you would need something like

    1. env.PATH = "$existing_path_wherever_it_may_come_from$:$individual_path$"

    But how to get the machines current path in question ?

    As you can see, there are a few quircks with your idea - or i did not follow you carefully enough.

    To monitor the rootserver, I want to use the Icinga2 agent and configured a portforwarding in my TP-Link router

    That would be needed if the external server should connect to the internal(master).

    But i bet that your master @ home is allowed to reach the internet, at least to reach the agents port 5665.

    Then you can omit the dyn dns service and the port forwarding at the tplink completely.

    In the internal masters zones.conf you would give the agents endpoint a host="" entry.

    In the external agents zones.conf you would not need a host="..."  neither for the agent nor for the master.

    Regarding the certificate setup,

    i would first validate both certificates at the master against the masters ca.crt:

    1. $ openssl verify -verbose -CAfile ca.crt master.crt
    2. master.crt: OK
    3. $ openssl verify -verbose -CAfile ca.crt agent.crt
    4. agent.crt: OK

    If that is fine, i would like to see the zones.conf file of both machines.

    Your downtime works just fine, i created it in my system.

    The reason you do not see it is that in the templates, you set the value of vars.wartung_downtime to "", but you should set it to something:

    vars.wartung_downtime="Yes. This Service needs the downtime".

    Your approach is exectly what i would do: preset the variable in the templates to *not* apply the downtime.

    But then, in the few services i need it i would define the variable to something like above.

    If you really want the downtime to be applied to every service, you can omit the variable completely and do something like:

    assign where

    as your apply rule.

    Dann solltest du das auch so machen.

    Wenn dich beim hostalive die rta times interessieren würden, ja dann...

    Aber wenn es dir bisher nur um "ist erreichbar oder nicht" ging dann bekommst du diese Info mit dem (wichtigeren) clustercheck zusätzlich.

    Do you recommend top-down config sync over top-down command endpoint in our scenario?

    i would opt for config sync.

    200 machines is not a small number so chances are that the one or other machine is not reachable via the network from time to time.

    Using config sync

    • all check results will be replayed after the network comes back.
    • you take the load from the master and satellites.
    • you are able to drive separate locations that might be prune to network drops.

    Using command_endpoint

    • Your machine stops checking if the master / satellite goes away.
    • With the check not executed, there is no backlog to replay.

    On the other hand, it is fine to start with command_endpoint and move an endpoint to config sync later.

    well, watching in the docs, it is a bug imho:

    Fully agreed. But that is my personal opinion.

    How and where would you suggest defining the host and service objects?

    Satellite related objects at the masters


    /etc/icinga2/zones.d/satellite/services.conf (at every apply statements 'assign', limit the zone to satellite)

    Master related objects either in conf.d or, to be consistent *and* to be prepared for a 2master HA setup:


    /etc/icinga2/zones.d/master/services.conf (at every apply statements 'assign', limit the zone to master)

    And objects you like to have in all zones below global-templates:



    but ymmv.

    I am not sure that this is what you are after.

    Setting the include on an agent results in config objects defined in the agents conf.d/ being (re-)evaluated.

    But normally you either like to push config objects from the master down to the agent or you push commands to be executed down to the agent.

    I personally feel you should remove that include line on the agent and create the missing objects (most likely the check_command) at the masters zones.d/[agentzone]/objects.conf file or in thezones.d/global-templates/objects.conf file so that it will be replicated down to the agent.

    Your solution may work fine in the moment, but if you create an object at the master later on that already exists in the agents conf.d directory,

    your agent will mis-validate the configuration and stop working.

    A zone for EVERY client? Seriously, who should admin this?

    A client is an endpoint object, a node in the cluster - and not a host object.

    And an endpoint object requires a zone.

    Read about clients in the documentation.

    If you want to use other agent based / agent less connections to run a check on a host or service, that is not bad.

    But the topic of your thread is the cluster logic.

    At most one more zone for all Clients makes sense for me

    Wont work.

    1. There is a limitation of 2 endpoints per zone, afaik.

    2. That would build a load sharing scenario - every client would receive all service and host objects for that zone and would agree with other endpoints in the same zone about who is checking what.

    Is this a bug

    My opinion is: Yes.

    But individual persons might have a different point of view.

    this makes the whole zones concept obsolete

    Think of an organisation that manages the IT for different customers.

    • Zones build a boundary of trust and as such separate the customers from each other by using strong encryption.
    • These different customers need different configuration objects - and zones enable exactly the distribution of this per-zone-information.
    • If one customer has multiple locations, zones provide you with the features of a HA- and load-sharing scenario.

    So, there might be persons in the wild that do not share above quote completely :/.

    when services are rolled out globally

    They are not.

    But the master believes they are. And sees "Fake late checks" for these.

    Check that on the satellites via icinga object list --type service --name SomeServiceNotInSatelliteZone.

    X clients - they are the monitored server.

    Ubuntu 16.04.2

    Icinga2 : r2.6.2-1

    Icingaweb2: 2.4.1

    So I build 3 Zones: global, master, satellite.

    ...and in addition a zone for each client controled by the satellites.

    An endpoint runs checks against host and service objects


    An endpoint tells another endpoint to execute a check_command

    In any case, the client would be an endpoint and thus needs to be a member in a zone.

    That was the answer for Problem #2.

    Every Server gets every service assigned.

    Master has the disk etc included ping, which is only defined in sat zone and the sat server have all master checks.

    Known problem.

    Fix it by filter the apply rules by the zone needed:

    1. apply Service "load" {
    2. import "generic-service"
    3. ...
    4. assign where"myzone"
    5. }

    You really did everything already i would have advised.

    Great !

    So what is left is not much:

    At the satellite, switch on the debug log: icinga2 feature enable debuglog && service icinga2 reload

    Wait until the service check has been executed and then see what icinga2 has run:

    grep -i 'running command' /var/log/icinga2/debuglog | grep -i dropped and post that here.

    disable the debuglog: icinga2 feature disable debuglog && service icinga2 reload

    Do the zone and endpoint need to be declared on both the master and the monitored host?

    Zone objects and endpoint objects go to zones.conf on every endpoint, be it a master, satellite or client.

    as you see, already answered... It has to be done manually, yes.

    /zones.d/myhost.conf <--- with a host object defined.


    Nice that you have a host object, but how can icinga2 tell into which zone it should replicate it ?

    Put it to /zones.d/myhost/myhost.conf so that icinga knows to which endpoints it should push the object,

    in this case these are all endpoints in zone "myhost".