1 # <a id="configuring-icinga2-first-steps"></a> Configuring Icinga 2: First Steps
3 This chapter provides an introduction into best practices with your Icinga 2 configuration.
4 The configuration files which are automatically created when installing the Icinga 2 packages
5 are a good way to start with Icinga 2.
7 If you're interested in a detailed explanation of each language feature used in those
8 configuration files, you can find more information in the [Language Reference](17-language-reference.md#language-reference)
11 ## <a id="configuration-best-practice"></a> Configuration Best Practice
13 If you are ready to configure additional hosts, services, notifications,
14 dependencies, etc., you should think about the requirements first and then
15 decide for a possible strategy.
17 There are many ways of creating Icinga 2 configuration objects:
19 * Manually with your preferred editor, for example vi(m), nano, notepad, etc.
20 * Generated by a [configuration management tool](13-addons.md#configuration-tools) such as Puppet, Chef, Ansible, etc.
21 * A configuration addon for Icinga 2 ([Icinga Director](https://github.com/Icinga/icingaweb2-module-director))
22 * A custom exporter script from your CMDB or inventory tool
25 In order to find the best strategy for your own configuration, ask yourself the following questions:
27 * Do your hosts share a common group of services (for example linux hosts with disk, load, etc. checks)?
28 * Only a small set of users receives notifications and escalations for all hosts/services?
30 If you can at least answer one of these questions with yes, look for the
31 [apply rules](3-monitoring-basics.md#using-apply) logic instead of defining objects on a per
32 host and service basis.
34 * You are required to define specific configuration for each host/service?
35 * Does your configuration generation tool already know about the host-service-relationship?
37 Then you should look for the object specific configuration setting `host_name` etc. accordingly.
39 Finding the best files and directory tree for your configuration is up to you. Make sure that
40 the [icinga2.conf](4-configuring-icinga-2.md#icinga2-conf) configuration file includes them,
43 * tree-based on locations, hostgroups, specific host attributes with sub levels of directories.
44 * flat `hosts.conf`, `services.conf`, etc. files for rule based configuration.
45 * generated configuration with one file per host and a global configuration for groups, users, etc.
46 * one big file generated from an external application (probably a bad idea for maintaining changes).
49 In either way of choosing the right strategy you should additionally check the following:
51 * Are there any specific attributes describing the host/service you could set as `vars` custom attributes?
52 You can later use them for applying assign/ignore rules, or export them into external interfaces.
53 * Put hosts into hostgroups, services into servicegroups and use these attributes for your apply rules.
54 * Use templates to store generic attributes for your objects and apply rules making your configuration more readable.
55 Details can be found in the [using templates](3-monitoring-basics.md#object-inheritance-using-templates) chapter.
56 * Apply rules may overlap. Keep a central place (for example, [services.conf](4-configuring-icinga-2.md#services-conf) or [notifications.conf](4-configuring-icinga-2.md#notifications-conf)) storing
57 the configuration instead of defining apply rules deep in your configuration tree.
58 * Every plugin used as check, notification or event command requires a `Command` definition.
59 Further details can be looked up in the [check commands](3-monitoring-basics.md#check-commands) chapter.
61 If you are planning to use a distributed monitoring setup with master, satellite and client installations
62 take the configuration location into account too. Everything configured on the master, synced to all other
63 nodes? Or any specific local configuration (e.g. health checks)?
67 If you happen to have further questions, do not hesitate to join the
68 [community support channels](https://www.icinga.com/community/get-involved/)
69 and ask community members for their experience and best practices.
71 ## <a id="configuring-icinga2-overview"></a> Configuration Overview
73 ### <a id="icinga2-conf"></a> icinga2.conf
75 An example configuration file is installed for you in `/etc/icinga2/icinga2.conf`.
77 Here's a brief description of the example configuration:
80 * Icinga 2 configuration file
81 * -- this is where you define settings for the Icinga application including
82 * which hosts/services to check.
84 * For an overview of all available configuration options please refer
85 * to the documentation that is distributed as part of Icinga 2.
88 Icinga 2 supports [C/C++-style comments](17-language-reference.md#comments).
91 * The constants.conf defines global constants.
93 include "constants.conf"
95 The `include` directive can be used to include other files.
98 * The zones.conf defines zones for a cluster setup.
99 * Not required for single instance setups.
103 The [Icinga Template Library](10-icinga-template-library.md#icinga-template-library) provides a set of common templates
104 and [CheckCommand](3-monitoring-basics.md#check-commands) definitions.
107 * The Icinga Template Library (ITL) provides a number of useful templates
108 * and command definitions.
109 * Common monitoring plugin command definitions are included separately.
113 include <plugins-contrib>
117 * This includes the Icinga 2 Windows plugins. These command definitions
118 * are required on a master node when a client is used as command endpoint.
120 include <windows-plugins>
123 * This includes the NSClient++ check commands. These command definitions
124 * are required on a master node when a client is used as command endpoint.
129 * The features-available directory contains a number of configuration
130 * files for features which can be enabled and disabled using the
131 * icinga2 feature enable / icinga2 feature disable CLI commands.
132 * These commands work by creating and removing symbolic links in
133 * the features-enabled directory.
135 include "features-enabled/*.conf"
137 This `include` directive takes care of including the configuration files for all
138 the features which have been enabled with `icinga2 feature enable`. See
139 [Enabling/Disabling Features](11-cli-commands.md#enable-features) for more details.
142 * The repository.d directory contains all configuration objects
143 * managed by the 'icinga2 repository' CLI commands.
145 include_recursive "repository.d"
147 This `include_recursive` directive is used for discovery of services on remote clients
148 and their generated configuration described in
149 [this chapter](6-distributed-monitoring.md#distributed-monitoring-bottom-up).
153 * Although in theory you could define all your objects in this file
154 * the preferred way is to create separate directories and files in the conf.d
155 * directory. Each of these files must have the file extension ".conf".
157 include_recursive "conf.d"
159 You can put your own configuration files in the [conf.d](4-configuring-icinga-2.md#conf-d) directory. This
160 directive makes sure that all of your own configuration files are included.
162 ### <a id="constants-conf"></a> constants.conf
164 The `constants.conf` configuration file can be used to define global constants.
166 By default, you need to make sure to set these constants:
168 * The `PluginDir` constant must be set to the path where the [Monitoring Project plugins](2-getting-started.md#setting-up-check-plugins) are installed.
169 This constant is used by a number of
170 [built-in check command definitions](10-icinga-template-library.md#plugin-check-commands).
171 * The `NodeName` constant defines your local node name. Should be set to FQDN which is the default
172 if not set. This constant is required for local host configuration, monitoring remote clients and
177 /* The directory which contains the plugins from the Monitoring Plugins project. */
178 const PluginDir = "/usr/lib64/nagios/plugins"
181 /* The directory which contains the Manubulon plugins.
182 * Check the documentation, chapter "SNMP Manubulon Plugin Check Commands", for details.
184 const ManubulonPluginDir = "/usr/lib64/nagios/plugins"
186 /* Our local instance name. By default this is the server's hostname as returned by `hostname --fqdn`.
187 * This should be the common name from the API certificate.
189 //const NodeName = "localhost"
191 /* Our local zone name. */
192 const ZoneName = NodeName
194 /* Secret key for remote node tickets */
195 const TicketSalt = ""
197 The `ZoneName` and `TicketSalt` constants are required for remote client
198 and distributed setups only.
200 ### <a id="conf-d"></a> The conf.d Directory
202 This directory contains example configuration which should help you get started
203 with monitoring the local host and its services. It is included in the
204 [icinga2.conf](4-configuring-icinga-2.md#icinga2-conf) configuration file by default.
206 It can be used as reference example for your own configuration strategy.
207 Just keep in mind to include the main directories in the
208 [icinga2.conf](4-configuring-icinga-2.md#icinga2-conf) file.
210 You are certainly not bound to it. Remove it if you prefer your own
211 way of deploying Icinga 2 configuration.
213 Further details on configuration best practice and how to build your
214 own strategy is described in [this chapter](4-configuring-icinga-2.md#configuration-best-practice).
216 Available configuration files which are installed by default:
218 * [hosts.conf](4-configuring-icinga-2.md#hosts-conf)
219 * [services.conf](4-configuring-icinga-2.md#services-conf)
220 * [users.conf](4-configuring-icinga-2.md#users-conf)
221 * [notifications.conf](4-configuring-icinga-2.md#notifications-conf)
222 * [commands.conf](4-configuring-icinga-2.md#commands-conf)
223 * [groups.conf](4-configuring-icinga-2.md#groups-conf)
224 * [templates.conf](4-configuring-icinga-2.md#templates-conf)
225 * [downtimes.conf](4-configuring-icinga-2.md#downtimes-conf)
226 * [timeperiods.conf](4-configuring-icinga-2.md#timeperiods-conf)
227 * [satellite.conf](4-configuring-icinga-2.md#satellite-conf)
228 * [api-users.conf](4-configuring-icinga-2.md#api-users-conf)
229 * [app.conf](4-configuring-icinga-2.md#app-conf)
231 #### <a id="hosts-conf"></a> hosts.conf
233 The `hosts.conf` file contains an example host based on your
234 `NodeName` setting in [constants.conf](4-configuring-icinga-2.md#constants-conf). You
235 can use global constants for your object names instead of string
238 The `import` keyword is used to import the `generic-host` template which
239 takes care of setting up the host check command to `hostalive`. If you
240 require a different check command, you can override it in the object definition.
242 The `vars` attribute can be used to define custom attributes which are available
243 for check and notification commands. Most of the [Plugin Check Commands](10-icinga-template-library.md#plugin-check-commands)
244 in the Icinga Template Library require an `address` attribute.
246 The custom attribute `os` is evaluated by the `linux-servers` group in
247 [groups.conf](4-configuring-icinga-2.md#groups-conf) making the local host a member.
249 The example host will show you how to
251 * define http vhost attributes for the `http` service apply rule defined
252 in [services.conf](4-configuring-icinga-2.md#services-conf).
253 * define disks (all, specific `/`) and their attributes for the `disk`
254 service apply rule defined in [services.conf](4-configuring-icinga-2.md#services-conf).
255 * define notification types (`mail`) and set the groups attribute. This
256 will be used by notification apply rules in [notifications.conf](notifications-conf).
258 If you've installed [Icinga Web 2](2-getting-started.md#setting-up-icingaweb2), you can
259 uncomment the http vhost attributes and reload Icinga 2. The apply
260 rules in [services.conf](4-configuring-icinga-2.md#services-conf) will automatically
261 generate a new service checking the `/icingaweb2` URI using the `http`
265 * Host definitions with object attributes
266 * used for apply rules for Service, Notification,
267 * Dependency and ScheduledDowntime objects.
269 * Tip: Use `icinga2 object list --type Host` to
270 * list all host objects after running
271 * configuration validation (`icinga2 daemon -C`).
275 * This is an example host based on your
276 * local host's FQDN. Specify the NodeName
277 * constant in `constants.conf` or use your
278 * own description, e.g. "db-host-1".
281 object Host NodeName {
282 /* Import the default host template defined in `templates.conf`. */
283 import "generic-host"
285 /* Specify the address attributes for checks e.g. `ssh` or `http`. */
286 address = "127.0.0.1"
289 /* Set custom attribute `os` for hostgroup assignment in `groups.conf`. */
292 /* Define http vhost attributes for service apply rules in `services.conf`. */
293 vars.http_vhosts["http"] = {
296 /* Uncomment if you've sucessfully installed Icinga Web 2. */
297 //vars.http_vhosts["Icinga Web 2"] = {
298 // http_uri = "/icingaweb2"
301 /* Define disks and attributes for service apply rules in `services.conf`. */
302 vars.disks["disk"] = {
305 vars.disks["disk /"] = {
306 disk_partitions = "/"
309 /* Define notification mail attributes for notification apply rules in `notifications.conf`. */
310 vars.notification["mail"] = {
311 /* The UserGroup `icingaadmins` is defined in `users.conf`. */
312 groups = [ "icingaadmins" ]
316 This is only the host object definition. Now we'll need to make sure that this
317 host and your additional hosts are getting [services](4-configuring-icinga-2.md#services-conf) applied.
321 > If you don't understand all the attributes and how to use [apply rules](17-language-reference.md#apply),
322 > don't worry -- the [monitoring basics](3-monitoring-basics.md#monitoring-basics) chapter will explain
325 #### <a id="services-conf"></a> services.conf
327 These service [apply rules](17-language-reference.md#apply) will show you how to monitor
328 the local host, but also allow you to re-use or modify them for
329 your own requirements.
331 You should define all your service apply rules in `services.conf`
332 or any other central location keeping them organized.
334 By default, the local host will be monitored by the following services
336 Service(s) | Applied on host(s)
337 --------------------------------------------|------------------------
338 `load`, `procs`, `swap`, `users`, `icinga` | The `NodeName` host only
339 `ping4`, `ping6` | All hosts with `address` resp. `address6` attribute
340 `ssh` | All hosts with `address` and `vars.os` set to `Linux`
341 `http`, optional: `Icinga Web 2` | All hosts with custom attribute `http_vhosts` defined as dictionary
342 `disk`, `disk /` | All hosts with custom attribute `disks` defined as dictionary
344 The Debian packages also include an additional `apt` service check applied to the local host.
346 The command object `icinga` for the embedded health check is provided by the
347 [Icinga Template Library (ITL)](10-icinga-template-library.md#icinga-template-library) while `http_ip`, `ssh`, `load`, `processes`,
348 `users` and `disk` are all provided by the [Plugin Check Commands](10-icinga-template-library.md#plugin-check-commands)
349 which we enabled earlier by including the `itl` and `plugins` configuration file.
352 Example `load` service apply rule:
354 apply Service "load" {
355 import "generic-service"
357 check_command = "load"
359 /* Used by the ScheduledDowntime apply rule in `downtimes.conf`. */
360 vars.backup_downtime = "02:00-03:00"
362 assign where host.name == NodeName
365 The `apply` keyword can be used to create new objects which are associated with
366 another group of objects. You can `import` existing templates, define (custom)
369 The custom attribe `backup_downtime` is defined to a specific timerange string.
370 This variable value will be used for applying a `ScheduledDowntime` object to
371 these services in [downtimes.conf](4-configuring-icinga-2.md#downtimes-conf).
373 In this example the `assign where` condition is a boolean expression which is
374 evaluated for all objects of type `Host` and a new service with name "load"
375 is created for each matching host. [Expression operators](17-language-reference.md#expression-operators)
376 may be used in `assign where` conditions.
378 Multiple `assign where` condition can be combined with `AND` using the `&&` operator
379 as shown in the `ssh` example:
381 apply Service "ssh" {
382 import "generic-service"
384 check_command = "ssh"
386 assign where host.address && host.vars.os == "Linux"
389 In this example, the service `ssh` is applied to all hosts having the `address`
390 attribute defined `AND` having the custom attribute `os` set to the string
392 You can modify this condition to match multiple expressions by combinding `AND`
393 and `OR` using `&&` and `||` [operators](17-language-reference.md#expression-operators), for example
394 `assign where host.address && (vars.os == "Linux" || vars.os == "Windows")`.
397 A more advanced example is shown by the `http` and `disk` service apply
398 rules. While one `apply` rule for `ssh` will only create a service for matching
399 hosts, you can go one step further: Generate apply rules based on array items
400 or dictionary key-value pairs.
402 The idea is simple: Your host in [hosts.conf](4-configuring-icinga-2.md#hosts-conf) defines the
403 `disks` dictionary as custom attribute in `vars`.
405 Remember the example from [hosts.conf](4-configuring-icinga-2.md#hosts-conf):
408 /* Define disks and attributes for service apply rules in `services.conf`. */
409 vars.disks["disk"] = {
412 vars.disks["disk /"] = {
418 This dictionary contains multiple service names we want to monitor. `disk`
419 should just check all available disks, while `disk /` will pass an additional
420 parameter `disk_partition` to the check command.
422 You'll recognize that the naming is important -- that's the very same name
423 as it is passed from a service to a check command argument. Read about services
424 and passing check commands in [this chapter](3-monitoring-basics.md#command-passing-parameters).
426 Using `apply Service for` omits the service name, it will take the key stored in
427 the `disk` variable in `key => config` as new service object name.
429 The `for` keyword expects a loop definition, for example `key => value in dictionary`
430 as known from Perl and other scripting languages.
432 Once defined like this, the `apply` rule defined below will do the following:
434 * only match hosts with `host.vars.disks` defined through the `assign where` condition
435 * loop through all entries in the `host.vars.disks` dictionary. That's `disk` and `disk /` as keys.
436 * call `apply` on each, and set the service object name from the provided key
437 * inside apply, the `generic-service` template is imported
438 * defining the [disk](10-icinga-template-library.md#plugin-check-command-disk) check command requiring command arguments like `disk_partition`
439 * adding the `config` dictionary items to `vars`. Simply said, there's now `vars.disk_partition` defined for the
442 Configuration example:
444 apply Service for (disk => config in host.vars.disks) {
445 import "generic-service"
447 check_command = "disk"
452 A similar example is used for the `http` services. That way you can make your
453 host the information provider for all apply rules. Define them once, and only
456 Look into [notifications.conf](4-configuring-icinga-2.md#notifications-conf) how this technique is used
457 for applying notifications to hosts and services using their type and user
460 Don't forget to install the [check plugins](2-getting-started.md#setting-up-check-plugins) required by
461 the hosts and services and their check commands.
463 Further details on the monitoring configuration can be found in the
464 [monitoring basics](3-monitoring-basics.md#monitoring-basics) chapter.
466 #### <a id="users-conf"></a> users.conf
468 Defines the `icingaadmin` User and the `icingaadmins` UserGroup. The latter is used in
469 [hosts.conf](4-configuring-icinga-2.md#hosts-conf) for defining a custom host attribute later used in
470 [notifications.conf](4-configuring-icinga-2.md#notifications-conf) for notification apply rules.
472 object User "icingaadmin" {
473 import "generic-user"
475 display_name = "Icinga 2 Admin"
476 groups = [ "icingaadmins" ]
478 email = "icinga@localhost"
481 object UserGroup "icingaadmins" {
482 display_name = "Icinga 2 Admin Group"
486 #### <a id="notifications-conf"></a> notifications.conf
488 Notifications for check alerts are an integral part or your
489 Icinga 2 monitoring stack.
491 The examples in this file define two notification apply rules for hosts and services.
492 Both `apply` rules match on the same condition: They are only applied if the
493 nested dictionary attribute `notification.mail` is set.
495 Please note that the `to` keyword is important in [notification apply rules](3-monitoring-basics.md#using-apply-notifications)
496 defining whether these notifications are applies to hosts or services.
497 The `import` keyword imports the specific mail templates defined in [templates.conf](4-configuring-icinga-2.md#templates-conf).
499 The `interval` attribute is not explicitly set -- it [defaults to 30 minutes](9-object-types.md#objecttype-notification).
501 By setting the `user_groups` to the value provided by the
502 respective [host.vars.notification.mail](4-configuring-icinga-2.md#hosts-conf) attribute we'll
503 implicitely use the `icingaadmins` UserGroup defined in [users.conf](4-configuring-icinga-2.md#users-conf).
505 apply Notification "mail-icingaadmin" to Host {
506 import "mail-host-notification"
508 user_groups = host.vars.notification.mail.groups
509 users = host.vars.notification.mail.users
511 assign where host.vars.notification.mail
514 apply Notification "mail-icingaadmin" to Service {
515 import "mail-service-notification"
517 user_groups = host.vars.notification.mail.groups
518 users = host.vars.notification.mail.users
520 assign where host.vars.notification.mail
523 More details on defining notifications and their additional attributes such as
524 filters can be read in [this chapter](3-monitoring-basics.md#notifications).
526 #### <a id="commands-conf"></a> commands.conf
528 This is the place where your own command configuration can be defined. By default
529 only the notification commands used by the notification templates defined in [templates.conf](4-configuring-icinga-2.md#templates-conf).
531 You can freely customize these notification commands, and adapt them for your needs.
532 Read more on that topic [here](3-monitoring-basics.md#notification-commands).
534 #### <a id="groups-conf"></a> groups.conf
536 The example host defined in [hosts.conf](hosts-conf) already has the
537 custom attribute `os` set to `Linux` and is therefore automatically
538 a member of the host group `linux-servers`.
540 This is done by using the [group assign](17-language-reference.md#group-assign) expressions similar
541 to previously seen [apply rules](3-monitoring-basics.md#using-apply).
543 object HostGroup "linux-servers" {
544 display_name = "Linux Servers"
546 assign where host.vars.os == "Linux"
549 object HostGroup "windows-servers" {
550 display_name = "Windows Servers"
552 assign where host.vars.os == "Windows"
555 Service groups can be grouped together by similar pattern matches.
556 The [match function](18-library-reference.md#global-functions-match) expects a wildcard match string
557 and the attribute string to match with.
559 object ServiceGroup "ping" {
560 display_name = "Ping Checks"
562 assign where match("ping*", service.name)
565 object ServiceGroup "http" {
566 display_name = "HTTP Checks"
568 assign where match("http*", service.check_command)
571 object ServiceGroup "disk" {
572 display_name = "Disk Checks"
574 assign where match("disk*", service.check_command)
578 #### <a id="templates-conf"></a> templates.conf
580 Most of the example configuration objects use generic global templates by
583 template Host "generic-host" {
584 max_check_attempts = 5
588 check_command = "hostalive"
591 template Service "generic-service" {
592 max_check_attempts = 3
597 The `hostalive` check command is part of the
598 [Plugin Check Commands](10-icinga-template-library.md#plugin-check-commands).
601 template Notification "mail-host-notification" {
602 command = "mail-host-notification"
604 states = [ Up, Down ]
605 types = [ Problem, Acknowledgement, Recovery, Custom,
606 FlappingStart, FlappingEnd,
607 DowntimeStart, DowntimeEnd, DowntimeRemoved ]
612 template Notification "mail-service-notification" {
613 command = "mail-service-notification"
615 states = [ OK, Warning, Critical, Unknown ]
616 types = [ Problem, Acknowledgement, Recovery, Custom,
617 FlappingStart, FlappingEnd,
618 DowntimeStart, DowntimeEnd, DowntimeRemoved ]
623 More details on `Notification` object attributes can be found [here](9-object-types.md#objecttype-notification).
626 #### <a id="downtimes-conf"></a> downtimes.conf
628 The `load` service apply rule defined in [services.conf](4-configuring-icinga-2.md#services-conf) defines
629 the `backup_downtime` custom attribute.
631 The [ScheduledDowntime](9-object-types.md#objecttype-scheduleddowntime) apply rule uses this attribute
632 to define the default value for the time ranges required for recurring downtime slots.
634 apply ScheduledDowntime "backup-downtime" to Service {
635 author = "icingaadmin"
636 comment = "Scheduled downtime for backup"
639 monday = service.vars.backup_downtime
640 tuesday = service.vars.backup_downtime
641 wednesday = service.vars.backup_downtime
642 thursday = service.vars.backup_downtime
643 friday = service.vars.backup_downtime
644 saturday = service.vars.backup_downtime
645 sunday = service.vars.backup_downtime
648 assign where service.vars.backup_downtime != ""
652 #### <a id="timeperiods-conf"></a> timeperiods.conf
654 This file contains the default timeperiod definitions for `24x7`, `9to5`
655 and `never`. TimePeriod objects are referenced by `*period`
656 objects such as hosts, services or notifications.
659 #### <a id="satellite-conf"></a> satellite.conf
661 Includes default templates and dependencies for
662 [monitoring remote clients](6-distributed-monitoring.md#distributed-monitoring)
663 using service discovery and
664 [config generation](6-distributed-monitoring.md#distributed-monitoring-bottom-up)
665 on the master. Can be ignored/removed on setups not using this feature.
668 Further details on the monitoring configuration can be found in the
669 [monitoring basics](3-monitoring-basics.md#monitoring-basics) chapter.
671 #### <a id="api-users-conf"></a> api-users.conf
673 Provides the default [ApiUser](9-object-types.md#objecttype-apiuser) object
674 named "root" for the [API authentication](12-icinga2-api.md#icinga2-api-authentication).
676 #### <a id="app-conf"></a> app.conf
678 Provides the default [IcingaApplication](9-object-types.md#objecttype-icingaapplication)
679 object named "app" for additional settings such as disabling notifications