Popular Icinga2 stacks?

stack
(Tim Johnston) #1

Every monitoring solution will have a bunch of supporting components (DB, dashboarding, log analysis, etc), and there are lots of options available. I see lots of comparisons of components (Icinga vs Nagios, Grafana vs Kibana) but not full solutions (“stacks”).

What are some common Icinga2 stacks in use?

For example, here you can see a bunch of different options (Icinga2+InfluxDB+Grafana vs Icinga2+ELK, etc).

  • Is that list fairly current?
  • What are you guys using?
  • Any articles you recommend with comparisons, tradeoffs etc?
(Rafael Voss) #2

I am using Icinga2+Director+InfluxDB+Grafana+ELK, but elk is in an early testing phase :slight_smile:

I tried graphite with wisperdb for performancedata, but whisperdb started to hate me after some time, so I switched to influxdb+grafana. For me influx/grafana looks a lot more modern and configuration syntax feels more straight forward to me. Also i like the influxdb documentation a lot more.

I would select the components that fit best with your experience, I am pretty sure you will find some graphite lovers here too :slight_smile:

1 Like
#3

Hi,

Icinga 2 has a bunch of integrations possibilities, you can get an overview here: https://www.icinga.com/products/integrations/.

The vagrant boxes are also a good starting point to get an overview what is possible and to get a first impression. Since you asked, yes the boxes are current.

The integrations highly depends on your environment, there is no universal Icinga 2 stack that works everywhere or makes sense to use everywhere. If you already have one of the featured integration solution on hand and get used to it, stay with it and integrate it in Icinga 2.

For example: In our work environment we use Icinga 2 with InfluxDB and Grafana, since we already had those solutions and knew how they work. Deploying Graphite for the single purpose to use it with Icinga 2 wouldn’t make sense in such a situation, since it would take time to get used to it and you need to maintain the Graphite instance as well as the existing InfluxDB & Grafana instances.

Therefore asking for a comparison between the integration solutions is not a simple question. :slight_smile:

Do you have any of the integration solutions on hand? How big do you plan to build your environment?

Best regards
Michael

1 Like
(Tim Johnston) #4

Cool. If you don’t mind me asking, why is ELK part of that?
Is it covering logging-specific stuff (through logstash)? Or are you evaluating it against the others?

In my setup, I’m leaning toward protototyping Icinga2+InfluxDB+Grafana. My immediate need is monitoring and dashboarding time series performance data. Of course, I might take on logging-related stuff later…

(Rafael Voss) #5

Yes iam covering log specifiv stuff.

  • SYSLOG
  • EVENTLOG
  • SNMP TRAPS

I am checking for some high prio logentires directly with Icinga and planing to use some passive Checks for traps and so on…

(Michael Friedrich) #6

I wouldn’t say that there’s a typical Icinga stack one generally would recommend. I wouldn’t start comparing different stacks to each other, but the tools solving a specific purpose underneath.

From a personal experience, which includes reading source code and (development) docs for their interfaces, I’d rank them the following within the Icinga context. You can find such opinionated talks also in the Icinga Camp and OSMC archives.

This year’s OSMC holds some interesting talks in this direction too.

Logs & Events

  • Elastic Stack. Their docs and product line evolved a lot, innovation and stability is key. Contra: Too many moving targets, languages, packages involved, also extra licenses for X-Pack.
  • Graylog. Great filters and correlation analysis, RBAC management with AD integration ootb. Contra: Hard to create dashboard widgets compared to Kibana, always depends on a specific ES backend version. Not so many developers like Elastic though.

Metrics

  • Graphite. Everything is Open Source, used widely and metrics can be inserted from the past (RRDTool cannot do that). Contra: Python and missing package dependencies make it hard to install. Scaling sometimes needs replacements with addons (carbon-cache-ng, etc.).
  • InfluxDB. Scaling and feature rich query language. Contra: Missing features just as partial deletes from data points. Distributed and large scale features are closed source enterprise only.
  • RRDTool. Fast and simple API. Contra: No REST-like interface to query, impossible to alter metrics after insert. PNP on top solves a few points.

In combination with the above, Grafana is the one and only interface I’d always use in combination.

Automation

  • Puppet. Descriptive and declarative language which defines the state, configuration and packages. Agent based with a central master. Contra: The weight between Open Source and Enterprise features is sometimes critical. They don’t like “The Foreman” project which competes with the PE console, adding a complete lifecycle system.
  • Ansible. Direct via SSH, no master/agent involved. Contra: Needs direct SSH access, config language is yaml.
  • Chef. (never used it, but I can see it being used for automated installs e.g. inside the GitLab package)
  • Salt. (never used it, but active minion management sounds like a nifty idea)

If you say Puppet or Ansible, I say “The Foreman” including lifecycle management and monitoring integration. Provision machines with KVM (OpenNebula, OpenStack, etc.) and have them organized in a frontend.

More Integrations with Icinga Web 2 Modules

  • Maps.
  • Graphite & Grafana.
  • Director for web based configuration and automated object sync rules from external sources (CMDB, AD, RDBMS, etc.)
  • Visualization with NagVis, when needed.
  • Custom Dashboards.

Conclusion

Always define the key elements your environment needs, and tackle the tools out there. There’s no single centric tool providing all of them, better weigh whether 3 tools and maintenance may solve the (manager’s) requirements.

3 Likes