Building Icinga2 Agent on AIX

This forum was archived to /woltlab and is now in read-only mode.
  • Hi all,

    in the meanwhile I succeeded in building the Icinga2 agent for AIX (without mysql/postgres support) but there is one last pain.

    The agent is running fine but is unable to write config updates from the master server.

    Precisely the agent is unable to create the needed directory structure beneath (/var/lib/icinga2/...)
    I defined a global zone named global on the icinga master and started the Icinga Agent on AIX

    $ icinga2 daemon


    [2017-05-29 14:55:58 CEST] warning/JsonRpcConnection: Error while processing message for identity 'master_server'

    Error: Function call 'opendir' for file '/var/lib/icinga2/api/zones/global' failed with error code 0, 'Error 0'

    Funny is the error code 0!

    $ ls -la /var/lib/icinga2/api/zones/

    total 0

    drwxr-xr-x 2 icinga icinga 256 May 29 14:41 .

    drwxr-xr-x 6 icinga icinga 256 May 29 10:24 ..

    Creating the directory for the global zone is not enough:

    $ mkdir /var/lib/icinga2/api/zones/global

    $ chown icinga:icinga /var/lib/icinga2/api/zones/global

    $ icinga2 daemon


    [2017-05-29 15:16:14 CEST] information/ApiListener: Updating configuration file: /var/lib/icinga2/api/zones/global//.timestamp

    [2017-05-29 15:16:14 CEST] information/ApiListener: Updating configuration file: /var/lib/icinga2/api/zones/global//_etc/linux_services.conf

    [2017-05-29 15:16:14 CEST] information/ApiListener: Updating configuration file: /var/lib/icinga2/api/zones/global//_etc/service_global.conf

    Looks good in the logfile but not in the filesystem:

    $ ls -la /var/lib/icinga2/api/zones/global

    drwxr-xr-x 2 icinga icinga 256 May 29 15:16 .

    drwxr-xr-x 3 icinga icinga 256 May 29 15:15 ..

    -rw-r--r-- 1 icinga icinga 17 May 29 15:16 .timestamp

    $ mkdir /var/lib/icinga2/api/zones/global/_etc

    $ chown icinga:icinga /var/lib/icinga2/api/zones/global/_etc

    $ icinga2 daemon


    [2017-05-29 15:20:58 CEST] information/ApiListener: Updating configuration file: /var/lib/icinga2/api/zones/global//_etc/linux_services.conf

    [2017-05-29 15:20:58 CEST] information/ApiListener: Updating configuration file: /var/lib/icinga2/api/zones/global//_etc/service_global.conf

    ls -l /var/lib/icinga2/api/zones/global//_etc/

    -rw-r--r-- 1 icinga icinga 171 May 29 15:20 linux_services.conf

    -rw-r--r-- 1 icinga icinga 640 May 29 15:20 service_global.conf

    Looks like AIX handles directories slightly different than Linux. :(

    Anyone an idea?

  • Hm, opendir() is called inside GlobRecursive() in apilistener-filesync.cpp

    ApiListener::LoadConfigDir() is invoked inside ApiListener::ConfigUpdateHandler() to determine whether there's existing configuration or any update required.

    Unfortunately we don't have any AIX systems at hand, so your best bet would be patching the source code and debug a little further with extended logging.

    The location where it fails is found in lib/base/utility.cpp -…lib/base/utility.cpp#L678

    In your case, dirp is a NULL pointer which causes the exception.

    It might be a permission issue on /var/lib/icinga2/api/zones/ itself. I don't know much about AIX, but are there any security mechanisms in place which could prevent the parent directory owner to write something into the directory?

  • There should be no more security mechanism as the usual owner & group membership.
    Creating the directories with the icinga user works just fine. But I'll check that.

    But why is the error Code 0? That sounds quite strange. Shouldn't it be somethin higher than 0?

    Would dirp not always be a NULL pointer if changes in zones are propagated by the master?

    When will mkdirp be called to create the directories for the zones?
    Shouldn't there be an exception if the mkdirp Fails?

  • dirp is the returned value from opendir() and that will be NULL if an error occurs (according to the man pages). errno will be populated with the error code, and is the thing you'll see in your logs.

    That is a function provided by gnu libc, which Icinga 2 relies on.

    It errors out on reading the directory, so the later steps with populating the directory are not happening. Enable the debug log to see the exact context what's going on.

    Mkdirp happens at a later stage. First off the zones directory is read on its own. It already fails in there, before even attempting to update/write any configuration received via json-rpc cluster message.

    You might test that with a sample program run as icinga user, like the man pages propose.…99/functions/opendir.html…de/Directory-Entries.html

    A five minute hack from man pages on my Fedora box:

    Try that on your box, also with different directories where it must fail as icinga or other user.

    Maybe there is a different handling/implementation on AIX for libc.

  • I checked for security and permission problems and let icinga run under root credentials. Same problem.

    The short opendir test was successful. No unexpected behaviour.

    Running icinga with debuglog enabled gave no hints about any issues.

    Last try was the removal of the leading / in apilistener-filesysnc.cpp

    Line 150: String tsPath = configDir + ".timestamp";

    As before the file is created but the problem still exists :(