Upgrade to 2.8 information/JsonRpcConnection: No messages for identity from client

xseth · Nov 28th 2017

Hi,

I followed the documentation for the migration between 2.7 to 2.8 but I don't know why I don't have response from master... No messages for identity

[2017-11-28 15:39:18 +0100] information/ApiListener: Sending replay log for endpoint 'xxxxxxxxx' in zone 'master'.

[2017-11-28 15:39:18 +0100] information/ApiListener: Finished sending replay log for endpoint 'xxxxxxxxx' in zone 'master'.

[2017-11-28 15:39:18 +0100] information/ApiListener: Finished syncing endpoint 'xxxxxxxxx' in zone 'master'.

[2017-11-28 15:39:48 +0100] information/WorkQueue: #5 (ApiListener, SyncQueue) items: 0, rate: 0.0166667/s (1/min 3/5min 6/15min);

[2017-11-28 15:39:48 +0100] information/WorkQueue: #4 (ApiListener, RelayQueue) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2017-11-28 15:39:48 +0100] information/WorkQueue: #9 (JsonRpcConnection, #1) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2017-11-28 15:39:48 +0100] information/WorkQueue: #8 (JsonRpcConnection, #0) items: 0, rate: 0/s (0/min 0/5min 0/15min);

[2017-11-28 15:40:18 +0100] information/JsonRpcConnection: No messages for identity 'xxxxxxxxx' have been received in the last 60 seconds.

[2017-11-28 15:40:18 +0100] warning/JsonRpcConnection: API client disconnected for identity 'xxxxxxxxx'

mcktr · Nov 28th 2017

Hi,

the Log is from your satellite or your master? Did you used the bottom up mode beforehand?

Please share your zone.conf from your master & satellite/client, as well as the versions that are included (# icinga2 --version) .

The network communication works, for example a ping from your master to your satellite and vice versa?

xseth · Nov 28th 2017

The log is from satellite client. Yes I followed this doc https://www.icinga.com/docs/ic…ved-bottom-up-client-mode

master zone.conf

Code

/*
* Endpoint and Zone configuration for a cluster setup
* This local example requires `NodeName` defined in
* constants.conf.
*/
object Endpoint NodeName {
host = NodeName
}
object Zone ZoneName {
endpoints = [ NodeName ]
}
/*
* Defines a global zone for distributed setups with masters,
* satellites and clients.
* This is required to sync configuration commands,
* templates, apply rules, etc. to satellite and clients.
* All nodes require the same configuration and must
* have `accept_config` enabled in the `api` feature.
*/
object Zone "global-templates" {
global = true
}
/*
* Defines a global zone for the Icinga Director.
* This is required to sync configuration commands,
* templates, apply rules, etc. to satellite and clients.
* All nodes require the same configuration and must
* have `accept_config` enabled in the `api` feature.
*/
object Zone "director-global" {
global = true
}
/*
* Read the documentation on how to configure
* a cluster setup with multiple zones.
*/
/*
object Endpoint "master.example.org" {
host = "master.example.org"
}
object Endpoint "satellite.example.org" {
host = "satellite.example.org"
}
object Zone "master" {
endpoints = [ "xxxxxxxxx" ]
}
object Zone "satellite" {
parent = "master"
endpoints = [ "satellite.example.org" ]
}
*/

Display More

Client zone.conf

Code

/*
* Generated by Icinga 2 node setup commands
* on 2017-08-01 12:40:15 +0200
*/
object Endpoint "xxxxxxxxx" {
host = "my_public_ipxxxxxxxxx"
port = "5665"
}
object Endpoint "clientxxxxxxxx" {
host = "client_public_ipxxxxxxxxx"
}
object Zone "master" {
endpoints = [ "xxxxxxxxx" ]
}
object Zone "clientxxxxxxxx" {
endpoints = [ "clientxxxxxxxx" ]
parent = "master"
}
object Zone "global-templates" {
global = true
}

Display More

Icinga2 version: r2.8.0-1

Communication work between master and client I don't have problem before upgrade to 2.8 but I wasn't using the api.

mcktr · Nov 29th 2017

The zone.conf on your master should contain the satellite/client. I recommend to read the Distributed Monitoring chapterin the documentation.

xseth · Nov 29th 2017

mcktr wrote:

The zone.conf on your master should contain the satellite/client. I recommend to read the Distributed Monitoring chapterin the documentation.

I don't have a satellite only master end clients...

Now I added endoint and zone on the master zone.conf

But I have this error from the client now

Code

[2017-11-29 13:20:23 +0100] information/JsonRpcConnection: Certificate request for CN 'xxxxxxxx' is pending. Waiting for approval.
[2017-11-29 13:20:23 +0100] warning/TlsStream: TLS stream was disconnected.
[2017-11-29 13:20:23 +0100] warning/JsonRpcConnection: API client disconnected for identity 'xxxxxxxx'
[2017-11-29 13:20:23 +0100] warning/ApiListener: Removing API client for endpoint 'xxxxxxxx'. 0 API clients left.

xseth · Nov 29th 2017

Ok now I reconfigured with node wizard my master and client.

But now with when I disable folder conf.d I have icingaweb2 totaly empty

How I can add my service from repository.d in globale-templates folder in zone.d ?

mcktr · Nov 29th 2017

There is a migration path from bottom up mode to top down mode: https://github.com/Icinga/icin…up-migration-to-top-down-

xseth · Nov 29th 2017

Thanks you

I recovered my services but on icingaweb2 the status stay in Pending... I don't know why

mcktr · Nov 29th 2017

The Icinga related communication between your master and client works? The certificate issue from your post above is resolved?

Please verfiy if your client accepts commands and/or configuration. Check /etc/icinga2/features-enabled/api.conf for accept_commands and accept_config. Pointer to the related documentation for those attributes: https://www.icinga.com/docs/ic…object-types/#apilistener

xseth · Nov 29th 2017

Yes I resolved the communication between the master and client. I reconfigured master node and client with node wizard and configure my endpoint en zone in zone.conf now the communication work.

I create simple check disk service on master to my client I can see this service on icingaweb2 with good free space, total storage.... but the status stay in Pending.

Yes, my client accept commands and config

mcktr · Nov 29th 2017

If I understood you correctly your check_disk service have check results, but it is still pending? I guess that the check results are old and coming from your bottom up mode setup.

Please show us the configuration of that service (# icinga2 object list --type Service --name "<Your Service Name here>").

xseth · Nov 29th 2017

Yes

Code

root@master-server:/# icinga2 object list --type Service --name "disk"
Object 'xxxxxxxxxxxxx.com!disk' of type 'Service':
% declared in '/etc/icinga2/zones.d/xxxxxxxxxxxxx.com/services.conf', lines 1:0-1:20
* __name = "xxxxxxxxxxxxx.com!disk"
* action_url = ""
* check_command = "disk"
% = modified in '/etc/icinga2/zones.d/xxxxxxxxxxxxx.com/services.conf', lines 4:3-4:24
* check_interval = 300
* check_period = ""
* check_timeout = null
* command_endpoint = ""
* display_name = "disk"
* enable_active_checks = true
* enable_event_handler = true
* enable_flapping = false
* enable_notifications = true
* enable_passive_checks = true
* enable_perfdata = true
* event_command = ""
* flapping_threshold = 0
* flapping_threshold_high = 30
* flapping_threshold_low = 25
* groups = [ ]
* host_name = "xxxxxxxxxxxxx.com"
% = modified in '/etc/icinga2/zones.d/xxxxxxxxxxxxx.com/services.conf', lines 2:3-2:48
* icon_image = ""
* icon_image_alt = ""
* max_check_attempts = 3
* name = "disk"
* notes = ""
* notes_url = ""
* package = "_etc"
* retry_interval = 60
* source_location
* first_column = 0
* first_line = 1
* last_column = 20
* last_line = 1
* path = "/etc/icinga2/zones.d/xxxxxxxxxxxxx.com/services.conf"
* templates = [ "disk" ]
% = modified in '/etc/icinga2/zones.d/xxxxxxxxxxxxx.com/services.conf', lines 1:0-1:20
* type = "Service"
* vars = null
* volatile = false
* zone = "xxxxxxxxxxxxx.com"

Display More

mcktr · Nov 30th 2017

After you configured the client, did you restart the icinga2 daemon?

The service looks good to me. Did you tried to force a check via "Check now"?

Please show us the current zone.conf from your master and client, also run the following command on both systems # icinga2 daemon --validate and post the output here. Please also search the Log on the client for entries that indicates that the config from master will be ignored.

Just to be sure, the hostname that you configured is identical with the hostname that you set on the master for the client?

xseth · Nov 30th 2017

Yes icinga2 daemon restarted and I tryed to force check now too.

zone.conf from master

Code

/*
* Generated by Icinga 2 node setup commands
* on 2017-11-29 13:41:19 +0100
*/
object Endpoint NodeName {
}
object Zone ZoneName {
endpoints = [ NodeName ]
}
object Zone "global-templates" {
global = true
}
object Zone "director-global" {
global = true
}
object Endpoint "xxxxxxxxxxxx.com" {
host = "xxx.xxx.xxx.xxx"
}
object Zone "xxxxxxxxxxxx.com" {
endpoints = [ "xxxxxxxxxxxx.com" ]
}

Display More

zone.conf from client

Code

/*
* Generated by Icinga 2 node setup commands
* on 2017-11-29 13:51:15 +0100
*/
object Endpoint "xxxxxx.xxx" {
host = "xxxxxx.xxx"
port = "5665"
}
object Zone "master" {
endpoints = [ "xxxxxx.xxx" ]
}
object Zone "global-templates" {
global = true
}
object Zone "director-global" {
global = true
}
object Endpoint NodeName {
}
object Zone ZoneName {
endpoints = [ NodeName ]
parent = "master"
}

Display More

icinga2 daemon --validate

Code

information/cli: Icinga application loader (version: r2.8.0-1)
information/cli: Loading configuration file(s).
information/ConfigItem: Committing config item(s).
information/ApiListener: My API identity: local.itc.ovh
warning/ApplyRule: Apply rule 'mail-icingaadmin' (in /etc/icinga2/zones.d/global-templates/notifications.conf: 11:1-11:45) for type 'Notification' does not match anywhere!
warning/ApplyRule: Apply rule 'mail-icingaadmin' (in /etc/icinga2/zones.d/global-templates/notifications.conf: 22:1-22:48) for type 'Notification' does not match anywhere!
warning/ApplyRule: Apply rule 'backup-downtime' (in /etc/icinga2/zones.d/global-templates/downtimes.conf: 5:1-5:52) for type 'ScheduledDowntime' does not match anywhere!
information/ConfigItem: Instantiated 1 ApiListener.
information/ConfigItem: Instantiated 4 Zones.
information/ConfigItem: Instantiated 2 Endpoints.
information/ConfigItem: Instantiated 1 FileLogger.
information/ConfigItem: Instantiated 2 NotificationCommands.
information/ConfigItem: Instantiated 212 CheckCommands.
information/ConfigItem: Instantiated 9 HostGroups.
information/ConfigItem: Instantiated 1 IcingaApplication.
information/ConfigItem: Instantiated 2 Hosts.
information/ConfigItem: Instantiated 1 UserGroup.
information/ConfigItem: Instantiated 2 Users.
information/ConfigItem: Instantiated 3 TimePeriods.
information/ConfigItem: Instantiated 3 Services.
information/ConfigItem: Instantiated 4 ServiceGroups.
information/ConfigItem: Instantiated 1 CheckerComponent.
information/ConfigItem: Instantiated 1 ExternalCommandListener.
information/ConfigItem: Instantiated 1 GraphiteWriter.
information/ConfigItem: Instantiated 1 InfluxdbWriter.
information/ConfigItem: Instantiated 1 PerfdataWriter.
information/ConfigItem: Instantiated 1 IdoMysqlConnection.
information/ConfigItem: Instantiated 1 NotificationComponent.
information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
information/cli: Finished validating the configuration file(s).

Display More

Yes the hostname is identical

If I check the log with tail -f /var/log/icinga2/icinga2.log from the client and force check now nothing happens in the log file...

mcktr · Nov 30th 2017

The parent zone for the client in the zones.conf on your master is missing.

Code

object Zone "xxxxxxxxxxxx.com" {
parent = ZoneName
endpoints = [ "xxxxxxxxxxxx.com" ]
}

Adjust your zones.conf, reload the icinga daemon on your master and now the check should leave the pending state.

xseth · Nov 30th 2017

mcktr wrote:

The parent zone for the client in the zones.conf on your master is missing.

Code

object Zone "xxxxxxxxxxxx.com" {

parent = ZoneName

endpoints = [ "xxxxxxxxxxxx.com" ]

}

Adjust your zones.conf, reload the icinga daemon on your master and now the check should leave the pending state.

it's work thanks you

Now I just have to do the other clients.

Share