Chef Icinga2 Best Practice Advice


(Taylor) #1

Hi All! I have lurked on this forum quite a bit for icinga2 and icingaweb2 configuration basics. For those basic issues it really helped out a lot! However, I am at the point where I need chef specific advice regarding automating chef environments/checks.

I am using the icinga2 core chef cookbook to monitor about 20 chef nodes at the moment, with plans on rolling it out to 100+. The consensus amongst my team regarding design, is having a client node’s main cookbook (i.e. our_app_cookbook) should control node attributes which then translate into what checks will be run on it. The try and explain more clearly this is how the workflow would look from the chef side:

icinga2_client_node

  1. our_app_cookbook installs icinga2-client and configures it to communicate with master
  2. our_app_cookbook sets node attributes to determine what services/checks need to be run on it
  3. it installs/configures our custom application

icinga2_master_node

  1. icinga2 and icingaweb2 already installed and running correctly
  2. chef-client converges and icinga2_environment custom resource searches our nodes for changes and finds the icinga2_client_node
  3. It extracts the attributes set on icinga2_client_node and applies the service(s)

Sorry if this is hard to understand, I am having a hard time trying to describe exactly what is needed as well. So please let me know if you have any questions about what I am trying to achieve.

Also I was thinking for this case a bottom up config sync type setup would work really well. Like a client node would dictate what checks it will run against itself and report the status to the master. I did see the bottom-up method was deprecated though…


(Taylor) #2

Incase anyone finds this I did find a way to accomplish this with Chef.

Client Nodes:

  • Set their own checks/services via node attributes.

Icinga2 Master Node:

  • Uses Chef::Search::Query.new() to parse node attributes and translate them into checks.
  • Heavy usage of icinga2_applyservice and applyservice template from node attributes.

Basically I had to write a handful of functions and custom resources to get this done, but in the end I totally automated getting services assigned to nodes that control their own checks.