CheckMK "users" page in WATO Configuration not working anymore

(Thomas) #1

Hello,

When I want to open the users page in WATO I receive an error “Internal error: ‘pid’” .

Here is the correspondig excerpt from /omd/sites/master/var/log/web.log

2019-07-19 09:16:56,101 [40] [cmk.web 1329] /master/check_mk/wato.py Internal error: Traceback (most recent call last):
File “/omd/sites/master/share/check_mk/web/htdocs/index.py”, line 115, in handler
handler()
File “/omd/sites/master/share/check_mk/web/htdocs/wato.py”, line 245, in page_handler
modeperms, modefunc = get_mode_function(current_mode)
File “/omd/sites/master/share/check_mk/web/htdocs/wato.py”, line 408, in get_mode_function
modefunc = mode_class.create_mode_function()
File “/omd/sites/master/share/check_mk/web/htdocs/wato.py”, line 437, in create_mode_function
mode_object = cls()
File “/omd/sites/master/share/check_mk/web/htdocs/wato.py”, line 11008, in init
self._job_snapshot = userdb.UserSyncBackgroundJob().get_status_snapshot()
File “/omd/sites/master/share/check_mk/web/htdocs/gui_background_job.py”, line 180, in get_status_snapshot
return GUIBackgroundStatusSnapshot(self)
File “/omd/sites/master/share/check_mk/web/htdocs/gui_background_job.py”, line 212, in init
self._job_status = background_job.get_status()
File “/omd/sites/master/share/check_mk/web/htdocs/background_job.py”, line 485, in get_status
p = psutil.Process(status[“pid”])
KeyError: ‘pid’

2019-07-19 09:17:01,066 [40] [cmk.web.job_manager 1310] Traceback (most recent call last):
File “/omd/sites/master/share/check_mk/web/htdocs/background_job.py”, line 622, in do_housekeeping
all_jobs.append((job_id, job_instances[job_id].get_status()))
File “/omd/sites/master/share/check_mk/web/htdocs/background_job.py”, line 485, in get_status
p = psutil.Process(status[“pid”])
KeyError: ‘pid’

2019-07-19 09:17:01,072 [40] [cmk.web 1310] Exception in cron job [execute_userdb_job]
Traceback (most recent call last):
File “/omd/sites/master/share/check_mk/web/htdocs/cron.py”, line 78, in page_run_cron
cron_job()
File “/omd/sites/master/share/check_mk/web/htdocs/userdb.py”, line 1303, in execute_userdb_job
if job.is_running():
File “/omd/sites/master/share/check_mk/web/htdocs/gui_background_job.py”, line 150, in is_running
return super(GUIBackgroundJobSnapshottedFunctions, self).is_running()
File “/omd/sites/master/share/check_mk/web/htdocs/background_job.py”, line 367, in is_running
job_status = self.get_status()
File “/omd/sites/master/share/check_mk/web/htdocs/background_job.py”, line 485, in get_status
p = psutil.Process(status[“pid”])
KeyError: ‘pid’

Unfortunately those errors do not really help me to debug this situation. Maybe someone here knows what to do already or what I could try next to track this problem?

Greetings and thanks in advance,
Thomas

(Philipp Näther) #2

This issue appeared early this year for another user on the mailing list, unfortunately there wasn’t any response or solution.
Updating your cmk verison probably will fix this issue.
Which cmk version do you run?
Are you using any kind of external user synchronization?

1 Like
(Thomas) #3

Thanks for your reply. Im running 1.5.0p9 already.

As I noticed that error on background jobs I just opened Background Jobs in WATO as well and received this URL: master/check_mk/index.py?start_url=%2Fmaster%2Fcheck_mk%2Fwato.py%3Ffolder%3D%26mode%3Dbackground_jobs_overview

Then I edited the Backup job there and modified it to be disabled. After that the Background Jobs page is broken as well.

Traceback (most recent call last):
File “/omd/sites/master/share/check_mk/web/htdocs/wato.py”, line 374, in page_handler
modefunc(“content”)
File “/omd/sites/master/share/check_mk/web/htdocs/wato.py”, line 446, in mode_function
return mode_object.handle_page()
File “/omd/sites/master/share/check_mk/web/htdocs/wato.py”, line 467, in handle_page
return self.page()
File “/omd/sites/master/share/check_mk/web/plugins/wato/background_job.py”, line 57, in page
if job_manager.get_running_job_ids(job_class):
File “/omd/sites/master/share/check_mk/web/htdocs/gui_background_job.py”, line 237, in get_running_job_ids
job_ids = super(GUIBackgroundJobManager, self).get_running_job_ids(job_class)
File “/omd/sites/master/share/check_mk/web/htdocs/background_job.py”, line 594, in get_running_job_ids
return [job_id for job_id in all_jobs if BackgroundJob(job_id, logger=self._logger).is_running()]
File “/omd/sites/master/share/check_mk/web/htdocs/background_job.py”, line 367, in is_running
job_status = self.get_status()
File “/omd/sites/master/share/check_mk/web/htdocs/background_job.py”, line 485, in get_status
p = psutil.Process(status[“pid”])
KeyError: ‘pid’

Though I can still access the background job trough this URL ending: master/check_mk/index.py?start_url=%2Fmaster%2Fcheck_mk%2Fwato.py%3Fmode%3Dbackup

There is no external user synchronization.

Greetings,
Thomas

(Philipp Näther) #4

Have you tried to restart your whole server? Maybe there is a process stuck without a PID file.

(Thomas) #5

Yes, I restarted the XEN VM already, that doesn´t help. I can not see any errors on hardware or OS level, there is absolutely nothing. This error obviously appeared randomly.

I really have no hint that it is not a software related issue to CheckMK.

(Philipp Näther) #6

1.5.0p19 is stable right now. Maybe you try this first before investigating further.

1 Like
(Thomas) #7

Thanks, I read to fast trough it and did not notice that the current version is p19, I read it as p9 as well :confused:

That update fixed the issue, everything is working again. We still do not know why this error occured but well, it´s gone. Thank you.

Greetings!