This part of the Icinga 2 documentation provides an overview of all the basic
monitoring concepts you need to know to run Icinga 2.
+Keep in mind these examples are made with a linux server in mind, if you are
+using Windows you will need to change the services accordingly. See the [ITL reference](7-icinga-template-library.md#windows-plugins)
+ for further information.
## <a id="hosts-services"></a> Hosts and Services
* Network services (HTTP, SMTP, SNMP, SSH, etc.)
* Printers
-* Switches / Routers
-* Temperature Sensors
+* Switches / routers
+* Temperature sensors
* Other local or network-accessible services
Host objects provide a mechanism to group services that are running
}
object Service "ping4" {
- host_name = "localhost"
+ host_name = "my-server1"
check_command = "ping4"
}
object Service "http" {
- host_name = "localhost"
- check_command = "http_ip"
+ host_name = "my-server1"
+ check_command = "http"
}
The example creates two services `ping4` and `http` which belong to the
It also specifies that the host should perform its own check using the `hostalive`
check command.
-The `address` custom attribute is used by check commands to determine which network
+The `address` attribute is used by check commands to determine which network
address is associated with the host object.
+Details on troubleshooting check problems can be found [here](16-troubleshooting.md#troubleshooting).
+
### <a id="host-states"></a> Host States
Hosts can be in any of the following states:
HARD | The host/service's state hasn't recently changed.
SOFT | The host/service has recently changed state and is being re-checked.
+### <a id="host-service-checks"></a> Host and Service Checks
+
+Hosts and services determine their state by running checks in a regular interval.
+
+ object Host "router" {
+ check_command = "hostalive"
+ address = "10.0.0.1"
+ }
+
+The `hostalive` command is one of several built-in check commands. It sends ICMP
+echo requests to the IP address specified in the `address` attribute to determine
+whether a host is online.
+
+A number of other [built-in check commands](7-icinga-template-library.md#plugin-check-commands) are also
+available. In addition to these commands the next few chapters will explain in
+detail how to set up your own check commands.
-## <a id="using-templates"></a> Using Templates
+
+## <a id="object-inheritance-using-templates"></a> Templates
Templates may be used to apply a set of identical attributes to more than one
object:
enable_perfdata = true
}
- object Service "ping4" {
+ apply Service "ping4" {
import "generic-service"
- host_name = "localhost"
check_command = "ping4"
+
+ assign where host.address
}
- object Service "ping6" {
+ apply Service "ping6" {
import "generic-service"
- host_name = "localhost"
check_command = "ping6"
+
+ assign where host.address6
}
+
In this example the `ping4` and `ping6` services inherit properties from the
template `generic-service`.
Objects as well as templates themselves can import an arbitrary number of
-templates. Attributes inherited from a template can be overridden in the
+other templates. Attributes inherited from a template can be overridden in the
object if necessary.
-## <a id="using-apply"></a> Apply objects based on rules
+You can also import existing non-template objects. Note that templates
+and objects share the same namespace, i.e. you can't define a template
+that has the same name like an object.
-Instead of assigning each object (`Service`, `Notification`, `Dependency`, `ScheduledDowntime`)
-based on attribute identifiers for example `host_name` objects can be [applied](#apply).
- apply Service "ping4" {
- import "generic-service"
+## <a id="custom-attributes"></a> Custom Attributes
- check_command = "ping4"
+In addition to built-in attributes you can define your own attributes:
- assign where "generic-host" in host.templates
- ignore where !host.address
+ object Host "localhost" {
+ vars.ssh_port = 2222
}
-In this example the `ping4` service will be created as object for all hosts importing
-the `generic-host` template. If the `address` attribute is not set, the host will be
-ignored.
+Valid values for custom attributes include:
-## <a id="groups"></a> Groups
+* [Strings](19-language-reference.md#string-literals), [numbers](19-language-reference.md#numeric-literals) and [booleans](19-language-reference.md#boolean-literals)
+* [Arrays](19-language-reference.md#array) and [dictionaries](19-language-reference.md#dictionary)
+* [Functions](3-monitoring-basics.md#custom-attributes-functions)
-Groups are used for combining hosts, services, and users into
-accessible configuration attributes and views in external (web)
-interfaces.
+### <a id="custom-attributes-functions"></a> Functions as Custom Attributes
-Group membership is defined at the respective object itself. If
-you have a hostgroup name `windows` for example, and want to assign
-specific hosts to this group for later viewing the group on your
-alert dashboard, first create the hostgroup:
+Icinga 2 lets you specify [functions](19-language-reference.md#functions) for custom attributes.
+The special case here is that whenever Icinga 2 needs the value for such a custom attribute it runs
+the function and uses whatever value the function returns:
- object HostGroup "windows" {
- display_name = "Windows Servers"
- }
+ object CheckCommand "random-value" {
+ import "plugin-check-command"
-Then add your hosts to this hostgroup
+ command = [ PluginDir + "/check_dummy", "0", "$text$" ]
- template Host "windows-server" {
- groups += [ "windows" ]
+ vars.text = {{ Math.random() * 100 }}
}
- object Host "mssql-srv1" {
- import "windows-server"
+This example uses the [abbreviated lambda syntax](19-language-reference.md#nullary-lambdas).
- vars.mssql_port = 1433
- }
+These functions have access to a number of variables:
- object Host "mssql-srv2" {
- import "windows-server"
+ Variable | Description
+ -------------|---------------
+ user | The User object (for notifications).
+ service | The Service object (for service checks/notifications/event handlers).
+ host | The Host object.
+ command | The command object (e.g. a CheckCommand object for checks).
- vars.mssql_port = 1433
- }
+Here's an example:
-This can be done for service and user groups the same way. Additionally
-the user groups are associated as attributes in `Notification` objects.
+ vars.text = {{ host.check_interval }}
- object UserGroup "windows-mssql-admins" {
- display_name = "Windows MSSQL Admins"
- }
+In addition to these variables the `macro` function can be used to retrieve the
+value of arbitrary macro expressions:
- template User "generic-windows-mssql-users" {
- groups += [ "windows-mssql-admins" ]
- }
+ vars.text = {{
+ if (macro("$address$") == "127.0.0.1") {
+ log("Running a check for localhost!")
+ }
- object User "win-mssql-noc" {
- import "generic-windows-mssql-users"
+ return "Some text"
+ }}
- email = "noc@example.com"
- }
+Acessing object attributes at runtime inside these functions is described in the
+[advanced topics](5-advanced-topics.md#access-object-attributes-at-runtime) chapter.
- object User "win-mssql-ops" {
- import "generic-windows-mssql-users"
+## <a id="runtime-macros"></a> Runtime Macros
- email = "ops@example.com"
- }
+Macros can be used to access other objects' attributes at runtime. For example they
+are used in command definitions to figure out which IP address a check should be
+run against:
-### <a id="groups"></a> Group Membership Assign
+ object CheckCommand "my-ping" {
+ import "plugin-check-command"
-If there is a certain number of hosts, services or users matching a pattern
-it's reasonable to assign the group object to these members.
-Details on the `assign where` syntax can be found [here]
+ command = [ PluginDir + "/check_ping", "-H", "$ping_address$" ]
- object HostGroup "mssql" {
- display_name = "MSSQL Servers"
- assign where host.vars.mssql_port
- }
+ arguments = {
+ "-w" = "$ping_wrta$,$ping_wpl$%"
+ "-c" = "$ping_crta$,$ping_cpl$%"
+ "-p" = "$ping_packets$"
+ }
-In this inherited example from above all hosts with the `var` `mssql_port`
-set will be added as members to the host group `mssql`.
+ vars.ping_address = "$address$"
+ vars.ping_wrta = 100
+ vars.ping_wpl = 5
-## <a id="timeperiods"></a> Time Periods
+ vars.ping_crta = 250
+ vars.ping_cpl = 10
-Time Periods define time ranges in Icinga where event actions are
-triggered, for example whether a service check is executed or not within
-the `check_period` attribute. Or a notification should be sent to
-users or not, filtered by the `period` and `notification_period`
-configuration attributes for `Notification` and `User` objects.
+ vars.ping_packets = 5
+ }
-> **Note**
->
-> If you are familar with Icinga 1.x - these time period definitions
-> are called `legacy timeperiods` in Icinga 2.
->
-> An Icinga 2 legacy timeperiod requires the `ITL` provided template
->`legacy-timeperiod`.
-
-The `TimePeriod` attribute `ranges` may contain multiple directives,
-including weekdays, days of the month, and calendar dates.
-These types may overlap/override other types in your ranges dictionary.
-
-The descending order of precedence is as follows:
-
-* Calendar date (2008-01-01)
-* Specific month date (January 1st)
-* Generic month date (Day 15)
-* Offset weekday of specific month (2nd Tuesday in December)
-* Offset weekday (3rd Monday)
-* Normal weekday (Tuesday)
-
-If you don't set any `check_period` or `notification_period` attribute
-on your configuration objects Icinga 2 assumes `24x7` as time period
-as shown below.
-
- object TimePeriod "24x7" {
- import "legacy-timeperiod"
-
- display_name = "Icinga 2 24x7 TimePeriod"
- ranges = {
- "monday" = "00:00-24:00"
- "tuesday" = "00:00-24:00"
- "wednesday" = "00:00-24:00"
- "thursday" = "00:00-24:00"
- "friday" = "00:00-24:00"
- "saturday" = "00:00-24:00"
- "sunday" = "00:00-24:00"
- }
+ object Host "router" {
+ check_command = "my-ping"
+ address = "10.0.0.1"
}
-If your operation staff should only be notified during workhours
-create a new timeperiod named `workhours` defining a work day from
-09:00 to 17:00.
+In this example we are using the `$address$` macro to refer to the host's `address`
+attribute.
- object TimePeriod "workhours" {
- import "legacy-timeperiod"
+We can also directly refer to custom attributes, e.g. by using `$ping_wrta$`. Icinga
+automatically tries to find the closest match for the attribute you specified. The
+exact rules for this are explained in the next section.
- display_name = "Icinga 2 8x5 TimePeriod"
- ranges = {
- "monday" = "09:00-17:00"
- "tuesday" = "09:00-17:00"
- "wednesday" = "09:00-17:00"
- "thursday" = "09:00-17:00"
- "friday" = "09:00-17:00"
- }
- }
-Use the `period` attribute to assign time periods to
-`Notification` and `Dependency` objects:
+### <a id="macro-evaluation-order"></a> Evaluation Order
- object Notification "mail" {
- import "generic-notification"
+When executing commands Icinga 2 checks the following objects in this order to look
+up macros and their respective values:
+
+1. User object (only for notifications)
+2. Service object
+3. Host object
+4. Command object
+5. Global custom attributes in the `Vars` constant
+
+This execution order allows you to define default values for custom attributes
+in your command objects.
+Here's how you can override the custom attribute `ping_packets` from the previous
+example:
+
+ object Service "ping" {
host_name = "localhost"
+ check_command = "my-ping"
- command = "mail-notification"
- users = [ "icingaadmin" ]
- period = "workhours"
+ vars.ping_packets = 10 // Overrides the default value of 5 given in the command
}
+If a custom attribute isn't defined anywhere an empty value is used and a warning is
+written to the Icinga 2 log.
+You can also directly refer to a specific attribute - thereby ignoring these evaluation
+rules - by specifying the full attribute name:
-## <a id="commands"></a> Commands
+ $service.vars.ping_wrta$
-Icinga 2 uses three different command object types to specify how
-checks should be performed, notifications should be sent and
-events should be handled.
+This retrieves the value of the `ping_wrta` custom attribute for the service. This
+returns an empty value if the service does not have such a custom attribute no matter
+whether another object such as the host has this attribute.
-### <a id="command-environment-variables"></a> Environment Variables for Commands
-Please check [Runtime Custom Attributes as Environment Variables](#runtime-custom-attribute-env-vars).
+### <a id="host-runtime-macros"></a> Host Runtime Macros
-### <a id="check-commands"></a> Check Commands
+The following host custom attributes are available in all commands that are executed for
+hosts or services:
-`CheckCommand` objects define the command line how a check is called.
+ Name | Description
+ -----------------------------|--------------
+ host.name | The name of the host object.
+ host.display_name | The value of the `display_name` attribute.
+ host.state | The host's current state. Can be one of `UNREACHABLE`, `UP` and `DOWN`.
+ host.state_id | The host's current state. Can be one of `0` (up), `1` (down) and `2` (unreachable).
+ host.state_type | The host's current state type. Can be one of `SOFT` and `HARD`.
+ host.check_attempt | The current check attempt number.
+ host.max_check_attempts | The maximum number of checks which are executed before changing to a hard state.
+ host.last_state | The host's previous state. Can be one of `UNREACHABLE`, `UP` and `DOWN`.
+ host.last_state_id | The host's previous state. Can be one of `0` (up), `1` (down) and `2` (unreachable).
+ host.last_state_type | The host's previous state type. Can be one of `SOFT` and `HARD`.
+ host.last_state_change | The last state change's timestamp.
+ host.downtime_depth | The number of active downtimes.
+ host.duration_sec | The time since the last state change.
+ host.latency | The host's check latency.
+ host.execution_time | The host's check execution time.
+ host.output | The last check's output.
+ host.perfdata | The last check's performance data.
+ host.last_check | The timestamp when the last check was executed.
+ host.check_source | The monitoring instance that performed the last check.
+ host.num_services | Number of services associated with the host.
+ host.num_services_ok | Number of services associated with the host which are in an `OK` state.
+ host.num_services_warning | Number of services associated with the host which are in a `WARNING` state.
+ host.num_services_unknown | Number of services associated with the host which are in an `UNKNOWN` state.
+ host.num_services_critical | Number of services associated with the host which are in a `CRITICAL` state.
-`CheckCommand` objects require the [ITL template](#itl-plugin-check-command)
-`plugin-check-command` to support native plugin based check methods.
+### <a id="service-runtime-macros"></a> Service Runtime Macros
-Unless you have done so already, download your check plugin and put it
-into the `PluginDir` directory. The following example uses the
-`check_disk` plugin shipped with the Monitoring Plugins package.
+The following service macros are available in all commands that are executed for
+services:
-The plugin path and all command arguments are made a list of
-double-quoted string arguments for proper shell escaping.
+ Name | Description
+ ---------------------------|--------------
+ service.name | The short name of the service object.
+ service.display_name | The value of the `display_name` attribute.
+ service.check_command | The short name of the command along with any arguments to be used for the check.
+ service.state | The service's current state. Can be one of `OK`, `WARNING`, `CRITICAL` and `UNKNOWN`.
+ service.state_id | The service's current state. Can be one of `0` (ok), `1` (warning), `2` (critical) and `3` (unknown).
+ service.state_type | The service's current state type. Can be one of `SOFT` and `HARD`.
+ service.check_attempt | The current check attempt number.
+ service.max_check_attempts | The maximum number of checks which are executed before changing to a hard state.
+ service.last_state | The service's previous state. Can be one of `OK`, `WARNING`, `CRITICAL` and `UNKNOWN`.
+ service.last_state_id | The service's previous state. Can be one of `0` (ok), `1` (warning), `2` (critical) and `3` (unknown).
+ service.last_state_type | The service's previous state type. Can be one of `SOFT` and `HARD`.
+ service.last_state_change | The last state change's timestamp.
+ service.downtime_depth | The number of active downtimes.
+ service.duration_sec | The time since the last state change.
+ service.latency | The service's check latency.
+ service.execution_time | The service's check execution time.
+ service.output | The last check's output.
+ service.perfdata | The last check's performance data.
+ service.last_check | The timestamp when the last check was executed.
+ service.check_source | The monitoring instance that performed the last check.
-Call the `check_disk` plugin with the `--help` parameter to see
-all available options. Our example defines warning (`-w`) and
-critical (`-c`) thresholds for the disk usage. Without any
-partition defined (`-p`) it will check all local partitions.
+### <a id="command-runtime-macros"></a> Command Runtime Macros
-Define the default check command custom attribute `disk_wfree` and `disk_cfree`
-freely definable naming schema) and their default threshold values. You can
-then use these custom attributes as runtime macros on the command line.
+The following custom attributes are available in all commands:
-The default custom attributes can be overridden by the custom attributes
-defined in the service using the check command `disk`. The custom attributes
-can also be inherited from a parent template using additive inheritance (`+=`).
+ Name | Description
+ -----------------------|--------------
+ command.name | The name of the command object.
- object CheckCommand "disk" {
- import "plugin-check-command"
+### <a id="user-runtime-macros"></a> User Runtime Macros
- command = [
- PluginDir + "/check_disk",
- "-w", "$disk_wfree$%",
- "-c", "$disk_cfree$%"
- ],
+The following custom attributes are available in all commands that are executed for
+users:
- vars.disk_wfree = 20
- vars.disk_cfree = 10
- }
+ Name | Description
+ -----------------------|--------------
+ user.name | The name of the user object.
+ user.display_name | The value of the display_name attribute.
-The host `localhost` with the service `disk` checks all disks with modified
-custom attributes (warning thresholds at `10%`, critical thresholds at `5%`
-free disk space).
+### <a id="notification-runtime-macros"></a> Notification Runtime Macros
- object Host "localhost" {
- import "generic-host"
+ Name | Description
+ -----------------------|--------------
+ notification.type | The type of the notification.
+ notification.author | The author of the notification comment, if existing.
+ notification.comment | The comment of the notification, if existing.
- address = "127.0.0.1"
- address6 = "::1"
- }
+### <a id="global-runtime-macros"></a> Global Runtime Macros
- object Service "disk" {
- import "generic-service"
+The following macros are available in all executed commands:
- host_name = "localhost"
- check_command = "disk"
+ Name | Description
+ -----------------------|--------------
+ icinga.timet | Current UNIX timestamp.
+ icinga.long_date_time | Current date and time including timezone information. Example: `2014-01-03 11:23:08 +0000`
+ icinga.short_date_time | Current date and time. Example: `2014-01-03 11:23:08`
+ icinga.date | Current date. Example: `2014-01-03`
+ icinga.time | Current time including timezone information. Example: `11:23:08 +0000`
+ icinga.uptime | Current uptime of the Icinga 2 process.
- vars.disk_wfree = 10
- vars.disk_cfree = 5
- }
+The following macros provide global statistics:
+ Name | Description
+ ----------------------------------|--------------
+ icinga.num_services_ok | Current number of services in state 'OK'.
+ icinga.num_services_warning | Current number of services in state 'Warning'.
+ icinga.num_services_critical | Current number of services in state 'Critical'.
+ icinga.num_services_unknown | Current number of services in state 'Unknown'.
+ icinga.num_services_pending | Current number of pending services.
+ icinga.num_services_unreachable | Current number of unreachable services.
+ icinga.num_services_flapping | Current number of flapping services.
+ icinga.num_services_in_downtime | Current number of services in downtime.
+ icinga.num_services_acknowledged | Current number of acknowledged service problems.
+ icinga.num_hosts_up | Current number of hosts in state 'Up'.
+ icinga.num_hosts_down | Current number of hosts in state 'Down'.
+ icinga.num_hosts_unreachable | Current number of unreachable hosts.
+ icinga.num_hosts_flapping | Current number of flapping hosts.
+ icinga.num_hosts_in_downtime | Current number of hosts in downtime.
+ icinga.num_hosts_acknowledged | Current number of acknowledged host problems.
-### <a id="notification-commands"></a> Notification Commands
-`NotificationCommand` objects define how notifications are delivered to external
-interfaces (E-Mail, XMPP, IRC, Twitter, etc).
+## <a id="using-apply"></a> Apply Rules
-`NotificationCommand` objects require the [ITL template](#itl-plugin-notification-command)
-`plugin-notification-command` to support native plugin-based notifications.
+Instead of assigning each object ([Service](6-object-types.md#objecttype-service),
+[Notification](6-object-types.md#objecttype-notification), [Dependency](6-object-types.md#objecttype-dependency),
+[ScheduledDowntime](6-object-types.md#objecttype-scheduleddowntime))
+based on attribute identifiers for example `host_name` objects can be [applied](19-language-reference.md#apply).
-Below is an example using runtime macros from Icinga 2 (such as `$service.output$` for
-the current check output) sending an email to the user(s) associated with the
-notification itself (`$user.email$`).
+Before you start using the apply rules keep the following in mind:
-If you want to specify default values for some of the custom attribute definitions,
-you can add a `vars` dictionary as shown for the `CheckCommand` object.
+* Define the best match.
+ * A set of unique [custom attributes](3-monitoring-basics.md#custom-attributes) for these hosts/services?
+ * Or [group](3-monitoring-basics.md#groups) memberships, e.g. a host being a member of a hostgroup, applying services to it?
+ * A generic pattern [match](19-language-reference.md#function-calls) on the host/service name?
+ * [Multiple expressions combined](3-monitoring-basics.md#using-apply-expressions) with `&&` or `||` [operators](19-language-reference.md#expression-operators)
+* All expressions must return a boolean value (an empty string is equal to `false` e.g.)
- object NotificationCommand "mail-service-notification" {
- import "plugin-notification-command"
+> **Note**
+>
+> You can set/override object attributes in apply rules using the respectively available
+> objects in that scope (host and/or service objects).
- command = [ SysconfDir + "/icinga2/scripts/mail-notification.sh" ]
+[Custom attributes](3-monitoring-basics.md#custom-attributes) can also store nested dictionaries and arrays. That way you can use them
+for not only matching for their existance or values in apply expressions, but also assign
+("inherit") their values into the generated objected from apply rules.
- env = {
- "NOTIFICATIONTYPE" = "$notification.type$"
- "SERVICEDESC" = "$service.name$"
- "HOSTALIAS" = "$host.display_name$",
- "HOSTADDRESS" = "$address$",
- "SERVICESTATE" = "$service.state$",
- "LONGDATETIME" = "$icinga.long_date_time$",
- "SERVICEOUTPUT" = "$service.output$",
- "NOTIFICATIONAUTHORNAME" = "$notification.author$",
- "NOTIFICATIONCOMMENT" = "$notification.comment$",
- "HOSTDISPLAYNAME" = "$host.display_name$",
- "SERVICEDISPLAYNAME" = "$service.display_name$",
- "USEREMAIL" = "$user.email$"
- }
- }
+* [Apply services to hosts](3-monitoring-basics.md#using-apply-services)
+* [Apply notifications to hosts and services](3-monitoring-basics.md#using-apply-notifications)
+* [Apply dependencies to hosts and services](3-monitoring-basics.md#using-apply-dependencies)
+* [Apply scheduled downtimes to hosts and services](3-monitoring-basics.md#using-apply-scheduledowntimes)
-The command attribute in the `mail-service-notification` command refers to the following
-shell script. The macros specified in the `env` array are exported
-as environment variables and can be used in the notification script:
+A more advanced example is using [apply with for loops on arrays or
+dictionaries](3-monitoring-basics.md#using-apply-for) for example provided by
+[custom atttributes](3-monitoring-basics.md#custom-attributes) or groups.
- #!/usr/bin/env bash
- template=$(cat <<TEMPLATE
- ***** Icinga *****
+> **Tip**
+>
+> Building configuration in that dynamic way requires detailed information
+> of the generated objects. Use the `object list` [CLI command](8-cli-commands.md#cli-command-object)
+> after successful [configuration validation](8-cli-commands.md#config-validation).
- Notification Type: $NOTIFICATIONTYPE
- Service: $SERVICEDESC
- Host: $HOSTALIAS
- Address: $HOSTADDRESS
- State: $SERVICESTATE
+### <a id="using-apply-expressions"></a> Apply Rules Expressions
- Date/Time: $LONGDATETIME
+You can use simple or advanced combinations of apply rule expressions. Each
+expression must evaluate into the boolean `true` value. An empty string
+will be for instance interpreted as `false`. In a similar fashion undefined
+attributes will return `false`.
- Additional Info: $SERVICEOUTPUT
+Returns `false`:
- Comment: [$NOTIFICATIONAUTHORNAME] $NOTIFICATIONCOMMENT
- TEMPLATE
- )
+ assign where host.vars.attribute_does_not_exist
- /usr/bin/printf "%b" $template | mail -s "$NOTIFICATIONTYPE - $HOSTDISPLAYNAME - $SERVICEDISPLAYNAME is $SERVICESTATE" $USEREMAIL
+Multiple `assign where` condition rows are evaluated as `OR` condition.
-While it's possible to specify the entire notification command right
-in the NotificationCommand object it is generally advisable to create a
-shell script in the `/etc/icinga2/scripts` directory and have the
-NotificationCommand object refer to that.
+You can combine multiple expressions for matching only a subset of objects. In some cases,
+you want to be able to add more than one assign/ignore where expression which matches
+a specific condition. To achieve this you can use the logical `and` and `or` operators.
-### <a id="event-commands"></a> Event Commands
-Unlike notifications event commands are called on every service state change
-if defined. Therefore the `EventCommand` object should define a command line
-evaluating the current service state and other service runtime attributes
-available through runtime vars. Runtime macros such as `$SERVICESTATETYPE$`
-and `$SERVICESTATE$` will be processed by Icinga 2 helping on fine-granular
-events being triggered.
+Match all `*mysql*` patterns in the host name and (`&&`) custom attribute `prod_mysql_db`
+matches the `db-*` pattern. All hosts with the custom attribute `test_server` set to `true`
+should be ignored, or any host name ending with `*internal` pattern.
-Common use case scenarios are a failing HTTP check requiring an immediate
-restart via event command, or if an application is locked and requires
-a restart upon detection.
+ object HostGroup "mysql-server" {
+ display_name = "MySQL Server"
-`EventCommand` objects require the ITL template `plugin-event-command`
-to support native plugin based checks.
+ assign where match("*mysql*", host.name) && match("db-*", host.vars.prod_mysql_db)
+ ignore where host.vars.test_server == true
+ ignore where match("*internal", host.name)
+ }
-When the event command is triggered on a service state change, it will
-send a check result using the `process_check_result` script forcibly
-changing the service state back to `OK` (`-r 0`) providing some debug
-information in the check output (`-o`).
+Similar example for advanced notification apply rule filters: If the service
+attribute `notes` contains the `has gold support 24x7` string `AND` one of the
+two condition passes: Either the `customer` host custom attribute is set to `customer-xy`
+`OR` the host custom attribute `always_notify` is set to `true`.
- object EventCommand "plugin-event-process-check-result" {
- import "plugin-event-command"
+The notification is ignored for services whose host name ends with `*internal`
+`OR` the `priority` custom attribute is [less than](19-language-reference.md#expression-operators) `2`.
- command = [
- PluginDir + "/process_check_result",
- "-H", "$host.name$",
- "-S", "$service.name$",
- "-c", LocalStateDir + "/run/icinga2/cmd/icinga2.cmd",
- "-r", "0",
- "-o", "Event Handler triggered in state '$service.state$' with output '$service.output$'."
- ]
+ template Notification "cust-xy-notification" {
+ users = [ "noc-xy", "mgmt-xy" ]
+ command = "mail-service-notification"
}
-### <a id="commands-arguments"></a> Command Arguments
+ apply Notification "notify-cust-xy-mysql" to Service {
+ import "cust-xy-notification"
-By defining a check command line using the `command` attribute Icinga 2
-will resolve all macros in the static string or array. Sometimes it is
-required to extend the arguments list based on a met condition evaluated
-at command execution. Or making arguments optional - only set if the
-macro value can be resolved by Icinga 2.
+ assign where match("*has gold support 24x7*", service.notes) && (host.vars.customer == "customer-xy" || host.vars.always_notify == true)
+ ignore where match("*internal", host.name) || (service.vars.priority < 2 && host.vars.is_clustered == true)
+ }
- object CheckCommand "check_http" {
- import "plugin-check-command"
- command = PluginDir + "/check_http"
- arguments = {
- "-H" = "$http_vhost$"
- "-I" = "$http_address$"
- "-u" = "$http_uri$"
- "-p" = "$http_port$"
- "-S" = {
- set_if = "$http_ssl$"
- }
- "-w" = "$http_warn_time$"
- "-c" = "$http_critical_time$"
+### <a id="using-apply-services"></a> Apply Services to Hosts
+
+The sample configuration already includes a detailed example in [hosts.conf](4-configuring-icinga-2.md#hosts-conf)
+and [services.conf](4-configuring-icinga-2.md#services-conf) for this use case.
+
+The example for `ssh` applies a service object to all hosts with the `address`
+attribute being defined and the custom attribute `os` set to the string `Linux` in `vars`.
+
+ apply Service "ssh" {
+ import "generic-service"
+
+ check_command = "ssh"
+
+ assign where host.address && host.vars.os == "Linux"
+ }
+
+
+Other detailed scenario examples are used in their respective chapters, for example
+[apply services with custom command arguments](3-monitoring-basics.md#command-passing-parameters).
+
+### <a id="using-apply-notifications"></a> Apply Notifications to Hosts and Services
+
+Notifications are applied to specific targets (`Host` or `Service`) and work in a similar
+manner:
+
+
+ apply Notification "mail-noc" to Service {
+ import "mail-service-notification"
+
+ user_groups = [ "noc" ]
+
+ assign where host.vars.notification.mail
+ }
+
+
+In this example the `mail-noc` notification will be created as object for all services having the
+`notification.mail` custom attribute defined. The notification command is set to `mail-service-notification`
+and all members of the user group `noc` will get notified.
+
+### <a id="using-apply-dependencies"></a> Apply Dependencies to Hosts and Services
+
+Detailed examples can be found in the [dependencies](3-monitoring-basics.md#dependencies) chapter.
+
+### <a id="using-apply-scheduledowntimes"></a> Apply Recurring Downtimes to Hosts and Services
+
+The sample configuration includes an example in [downtimes.conf](4-configuring-icinga-2.md#downtimes-conf).
+
+Detailed examples can be found in the [recurring downtimes](5-advanced-topics.md#recurring-downtimes) chapter.
+
+
+### <a id="using-apply-for"></a> Using Apply For Rules
+
+Next to the standard way of using [apply rules](3-monitoring-basics.md#using-apply)
+there is the requirement of generating apply rules objects based on set (array or
+dictionary).
+
+The sample configuration already includes a detailed example in [hosts.conf](4-configuring-icinga-2.md#hosts-conf)
+and [services.conf](4-configuring-icinga-2.md#services-conf) for this use case.
+
+Take the following example: A host provides the snmp oids for different service check
+types. This could look like the following example:
+
+ object Host "router-v6" {
+ check_command = "hostalive"
+ address6 = "::1"
+
+ vars.oids["if01"] = "1.1.1.1.1"
+ vars.oids["temp"] = "1.1.1.1.2"
+ vars.oids["bgp"] = "1.1.1.1.5"
+ }
+
+Now we want to create service checks for `if01` and `temp` but not `bgp`.
+Furthermore we want to pass the snmp oid stored as dictionary value to the
+custom attribute called `vars.snmp_oid` - this is the command argument required
+by the [snmp](7-icinga-template-library.md#plugin-check-command-snmp) check command.
+The service's `display_name` should be set to the identifier inside the dictionary.
+
+ apply Service for (identifier => oid in host.vars.oids) {
+ check_command = "snmp"
+ display_name = identifier
+ vars.snmp_oid = oid
+
+ ignore where identifier == "bgp" //don't generate service for bgp checks
+ }
+
+Icinga 2 evaluates the `apply for` rule for all objects with the custom attribute
+`oids` set. It then iterates over all list items inside the `for` loop and evaluates the
+`assign/ignore where` expressions. You can access the loop variable
+in these expressions, e.g. for ignoring certain values.
+In this example we'd ignore the `bgp` identifier and avoid generating an unwanted service.
+We could extend the configuration by also matching the `oid` value on certain regex/wildcard
+patterns for example.
+
+> **Note**
+>
+> You don't need an `assign where` expression only checking for existance
+> of the custom attribute.
+
+That way you'll save duplicated apply rules by combining them into one
+generic `apply for` rule generating the object name with or without a prefix.
+
+
+#### <a id="using-apply-for-custom-attribute-override"></a> Apply For and Custom Attribute Override
+
+Imagine a different more advanced example: You are monitoring your network device (host)
+with many interfaces (services). The following requirements/problems apply:
+
+* Each interface service check should be named with a prefix and a name defined in your host object (which could be generated from your CMDB, etc)
+* Each interface has its own vlan tag
+* Some interfaces have QoS enabled
+* Additional attributes such as `display_name` or `notes, `notes_url` and `action_url` must be
+dynamically generated
+
+
+Tip: Define the snmp community as global constant in your [constants.conf](4-configuring-icinga-2.md#constants-conf) file.
+
+ const IftrafficSnmpCommunity = "public"
+
+By defining the `interfaces` dictionary with three example interfaces on the `cisco-catalyst-6509-34`
+host object, you'll make sure to pass the [custom attribute](3-monitoring-basics.md#custom-attributes)
+storage required by the for loop in the service apply rule.
+
+ object Host "cisco-catalyst-6509-34" {
+ import "generic-host"
+ display_name = "Catalyst 6509 #34 VIE21"
+ address = "127.0.1.4"
+
+ /* "GigabitEthernet0/2" is the interface name,
+ * and key name in service apply for later on
+ */
+ vars.interfaces["GigabitEthernet0/2"] = {
+ /* define all custom attributes with the
+ * same name required for command parameters/arguments
+ * in service apply (look into your CheckCommand definition)
+ */
+ iftraffic_units = "g"
+ iftraffic_community = IftrafficSnmpCommunity
+ iftraffic_bandwidth = 1
+ vlan = "internal"
+ qos = "disabled"
+ }
+ vars.interfaces["GigabitEthernet0/4"] = {
+ iftraffic_units = "g"
+ //iftraffic_community = IftrafficSnmpCommunity
+ iftraffic_bandwidth = 1
+ vlan = "renote"
+ qos = "enabled"
+ }
+ vars.interfaces["MgmtInterface1"] = {
+ iftraffic_community = IftrafficSnmpCommunity
+ vlan = "mgmt"
+ interface_address = "127.99.0.100" #special management ip
}
+ }
- vars.http_address = "$address$"
- vars.http_ssl = false
+You can also omit the `"if-"` string, then all generated service names are directly
+taken from the `if_name` variable value.
+
+The config dictionary contains all key-value pairs for the specific interface in one
+loop cycle, like `iftraffic_units`, `vlan`, and `qos` for the specified interface.
+
+You can either map the custom attributes from the `interface_config` dictionary to
+local custom attributes stashed into `vars`. If the names match the required command
+argument parameters already (for example `iftraffic_units`), you could also add the
+`interface_config` dictionary to the `vars` dictionary using the `+=` operator.
+
+After `vars` is fully populated, all object attributes can be set calculated from
+provided host attributes. For strings, you can use string concatention with the `+` operator.
+
+You can also specifiy the display_name, check command, interval, notes, notes_url, action_url, etc.
+attributes that way. Attribute strings can be [concatenated](19-language-reference.md#expression-operators),
+for example for adding a more detailed service `display_name`.
+
+This example also uses [if conditions](19-language-reference.md#conditional-statements)
+if specific values are not set, adding a local default value.
+The other way around you can override specific custom attributes inherited from a service template,
+if set.
+
+ /* loop over the host.vars.interfaces dictionary
+ * for (key => value in dict) means `interface_name` as key
+ * and `interface_config` as value. Access config attributes
+ * with the indexer (`.`) character.
+ */
+ apply Service "if-" for (interface_name => interface_config in host.vars.interfaces) {
+ import "generic-service"
+ check_command = "iftraffic"
+ display_name = "IF-" + interface_name
+
+ /* use the key as command argument (no duplication of values in host.vars.interfaces) */
+ vars.iftraffic_interface = interface_name
+
+ /* map the custom attributes as command arguments */
+ vars.iftraffic_units = interface_config.iftraffic_units
+ vars.iftraffic_community = interface_config.iftraffic_community
+
+ /* the above can be achieved in a shorter fashion if the names inside host.vars.interfaces
+ * are the _exact_ same as required as command parameter by the check command
+ * definition.
+ */
+ vars += interface_config
+
+ /* set a default value for units and bandwidth */
+ if (interface_config.iftraffic_units == "") {
+ vars.iftraffic_units = "m"
+ }
+ if (interface_config.iftraffic_bandwidth == "") {
+ vars.iftraffic_bandwidth = 1
+ }
+ if (interface_config.vlan == "") {
+ vars.vlan = "not set"
+ }
+ if (interface_config.qos == "") {
+ vars.qos = "not set"
+ }
+
+ /* set the global constant if not explicitely
+ * not provided by the `interfaces` dictionary on the host
+ */
+ if (len(interface_config.iftraffic_community) == 0 || len(vars.iftraffic_community) == 0) {
+ vars.iftraffic_community = IftrafficSnmpCommunity
+ }
+
+ /* Calculate some additional object attributes after populating the `vars` dictionary */
+ notes = "Interface check for " + interface_name + " (units: '" + interface_config.iftraffic_units + "') in VLAN '" + vars.vlan + "' with ' QoS '" + vars.qos + "'"
+ notes_url = "http://foreman.company.com/hosts/" + host.name
+ action_url = "http://snmp.checker.company.com/" + host.name + "/if-" + interface_name
}
-The example shows the `check_http` check command defining the most common
-arguments. Each of them is optional by default and will be omitted if
-the value is not set. For example if the service calling the check command
-does not have `vars.http_port` set, it won't get added to the command
-line.
-If the `vars.http_ssl` custom attribute is set in the service, host or command
-object definition, Icinga 2 will add the `-S` argument based on the `set_if`
-option to the command line.
-That way you can use the `check_http` command definition for both, with and
-without SSL enabled checks saving you duplicated command definitions.
-Details on all available options can be found in the
-[CheckCommand object definition](#objecttype-checkcommand).
+This example makes use of the [check_iftraffic](https://exchange.icinga.org/exchange/iftraffic) plugin.
+The `CheckCommand` definition can be found in the
+[contributed plugin check commands](7-icinga-template-library.md#plugins-contrib-command-iftraffic)
+- make sure to include them in your [icinga2 configuration file](4-configuring-icinga-2.md#icinga2-conf).
+
+
+> **Tip**
+>
+> Building configuration in that dynamic way requires detailed information
+> of the generated objects. Use the `object list` [CLI command](8-cli-commands.md#cli-command-object)
+> after successful [configuration validation](8-cli-commands.md#config-validation).
+
+Verify that the apply-for-rule successfully created the service objects with the
+inherited custom attributes:
+
+ # icinga2 daemon -C
+ # icinga2 object list --type Service --name *catalyst*
+
+Object 'cisco-catalyst-6509-34!if-GigabitEthernet0/2' of type 'Service':
+ ......
+ * vars
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 59:3-59:26
+ * iftraffic_bandwidth = 1
+ * iftraffic_community = "public"
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 53:3-53:65
+ * iftraffic_interface = "GigabitEthernet0/2"
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 49:3-49:43
+ * iftraffic_units = "g"
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 52:3-52:57
+ * qos = "disabled"
+ * vlan = "internal"
+
+
+ Object 'cisco-catalyst-6509-34!if-GigabitEthernet0/4' of type 'Service':
+ ...
+ * vars
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 59:3-59:26
+ * iftraffic_bandwidth = 1
+ * iftraffic_community = "public"
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 53:3-53:65
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 79:5-79:53
+ * iftraffic_interface = "GigabitEthernet0/4"
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 49:3-49:43
+ * iftraffic_units = "g"
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 52:3-52:57
+ * qos = "enabled"
+ * vlan = "renote"
+
+ Object 'cisco-catalyst-6509-34!if-MgmtInterface1' of type 'Service':
+ ...
+ * vars
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 59:3-59:26
+ * iftraffic_bandwidth = 1
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 66:5-66:32
+ * iftraffic_community = "public"
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 53:3-53:65
+ * iftraffic_interface = "MgmtInterface1"
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 49:3-49:43
+ * iftraffic_units = "m"
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 52:3-52:57
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 63:5-63:30
+ * interface_address = "127.99.0.100"
+ * qos = "not set"
+ % = modified in '/etc/icinga2/conf.d/iftraffic.conf', lines 72:5-72:24
+ * vlan = "mgmt"
+
+
+### <a id="using-apply-object-attributes"></a> Use Object Attributes in Apply Rules
+
+Since apply rules are evaluated after the generic objects, you
+can reference existing host and/or service object attributes as
+values for any object attribute specified in that apply rule.
+
+ object Host "opennebula-host" {
+ import "generic-host"
+ address = "10.1.1.2"
+
+ vars.hosting["xyz"] = {
+ http_uri = "/shop"
+ customer_name = "Customer xyz"
+ customer_id = "7568"
+ support_contract = "gold"
+ }
+ vars.hosting["abc"] = {
+ http_uri = "/shop"
+ customer_name = "Customer xyz"
+ customer_id = "7568"
+ support_contract = "silver"
+ }
+ }
+
+ apply Service for (customer => config in host.vars.hosting) {
+ import "generic-service"
+ check_command = "ping4"
+
+ vars.qos = "disabled"
+
+ vars += config
+
+ vars.http_uri = "/" + vars.customer + "/" + config.http_uri
+
+ display_name = "Shop Check for " + vars.customer_name + "-" + vars.customer_id
+
+ notes = "Support contract: " + vars.support_contract + " for Customer " + vars.customer_name + " (" + vars.customer_id + ")."
+
+ notes_url = "http://foreman.company.com/hosts/" + host.name
+ action_url = "http://snmp.checker.company.com/" + host.name + "/" + vars.customer_id
+ }
+
+## <a id="groups"></a> Groups
+
+A group is a collection of similar objects. Groups are primarily used as a
+visualization aid in web interfaces.
+
+Group membership is defined at the respective object itself. If
+you have a hostgroup name `windows` for example, and want to assign
+specific hosts to this group for later viewing the group on your
+alert dashboard, first create a HostGroup object:
+
+ object HostGroup "windows" {
+ display_name = "Windows Servers"
+ }
+
+Then add your hosts to this group:
+
+ template Host "windows-server" {
+ groups += [ "windows" ]
+ }
+
+ object Host "mssql-srv1" {
+ import "windows-server"
+
+ vars.mssql_port = 1433
+ }
+
+ object Host "mssql-srv2" {
+ import "windows-server"
+
+ vars.mssql_port = 1433
+ }
+
+This can be done for service and user groups the same way:
+
+ object UserGroup "windows-mssql-admins" {
+ display_name = "Windows MSSQL Admins"
+ }
+
+ template User "generic-windows-mssql-users" {
+ groups += [ "windows-mssql-admins" ]
+ }
+
+ object User "win-mssql-noc" {
+ import "generic-windows-mssql-users"
+
+ email = "noc@example.com"
+ }
+
+ object User "win-mssql-ops" {
+ import "generic-windows-mssql-users"
+
+ email = "ops@example.com"
+ }
+
+### <a id="group-assign-intro"></a> Group Membership Assign
+
+Instead of manually assigning each object to a group you can also assign objects
+to a group based on their attributes:
+
+ object HostGroup "prod-mssql" {
+ display_name = "Production MSSQL Servers"
+
+ assign where host.vars.mssql_port && host.vars.prod_mysql_db
+ ignore where host.vars.test_server == true
+ ignore where match("*internal", host.name)
+ }
+
+In this example all hosts with the `vars` attribute `mssql_port`
+will be added as members to the host group `mssql`. However, all `*internal`
+hosts or with the `test_server` attribute set to `true` are not added to this
+group.
+
+Details on the `assign where` syntax can be found in the
+[Language Reference](19-language-reference.md#apply)
## <a id="notifications"></a> Notifications
Notifications for service and host problems are an integral part of your
monitoring setup.
+When a host or service is in a downtime, a problem has been acknowledged or
+the dependency logic determined that the host/service is unreachable, no
+notifications are sent. You can configure additional type and state filters
+refining the notifications being actually sent.
+
There are many ways of sending notifications, e.g. by e-mail, XMPP,
IRC, Twitter, etc. On its own Icinga 2 does not know how to send notifications.
Instead it relies on external mechanisms such as shell scripts to notify users.
+More notification methods are listed in the [addons and plugins](13-addons-plugins.md#notification-scripts-interfaces)
+chapter.
A notification specification requires one or more users (and/or user groups)
who will be notified in case of problems. These users must have all custom
email = "icinga@localhost"
}
-If you don't set the `states` and `types`
-configuration attributes for the `User` object, notifications for all states and types
-will be sent.
+If you don't set the `states` and `types` configuration attributes for the `User`
+object, notifications for all states and types will be sent.
+
+Details on troubleshooting notification problems can be found [here](16-troubleshooting.md#troubleshooting).
+
+> **Note**
+>
+> Make sure that the [notification](8-cli-commands.md#features) feature is enabled
+> in order to execute notification commands.
You should choose which information you (and your notified users) are interested in
case of emergency, and also which information does not provide any value to you and
your environment.
-An example notification command is explained [here](#notification-commands).
+An example notification command is explained [here](3-monitoring-basics.md#notification-commands).
You can add all shared attributes to a `Notification` template which is inherited
to the defined notifications. That way you'll save duplicated attributes in each
`Notification` object. Attributes can be overridden locally.
-
template Notification "generic-notification" {
interval = 15m
states = [ Warning, Critical, Unknown ]
types = [ Problem, Acknowledgement, Recovery, Custom, FlappingStart,
- FlappingEnd, DowntimeStart,DowntimeEnd, DowntimeRemoved ]
+ FlappingEnd, DowntimeStart, DowntimeEnd, DowntimeRemoved ]
period = "24x7"
}
-The time period `24x7` is shipped as example configuration with Icinga 2.
+The time period `24x7` is included as example configuration with Icinga 2.
Use the `apply` keyword to create `Notification` objects for your services:
- apply Notification "mail" to Service {
+ apply Notification "notify-cust-xy-mysql" to Service {
import "generic-notification"
- command = "mail-notification"
- users = [ "icingaadmin" ]
+ users = [ "noc-xy", "mgmt-xy" ]
- assign where service.name == "mysql"
+ assign where match("*has gold support 24x7*", service.notes) && (host.vars.customer == "customer-xy" || host.vars.always_notify == true
+ ignore where match("*internal", host.name) || (service.vars.priority < 2 && host.vars.is_clustered == true)
}
+
Instead of assigning users to notifications, you can also add the `user_groups`
attribute with a list of user groups to the `Notification` object. Icinga 2 will
send notifications to all group members.
+> **Note**
+>
+> Only users who have been notified of a problem before (`Warning`, `Critical`, `Unknown`
+> states for services, `Down` for hosts) will receive `Recovery` notifications.
+
### <a id="notification-escalations"></a> Notification Escalations
-When a problem notification is sent and a problem still exists after re-notification
+When a problem notification is sent and a problem still exists at the time of re-notification
you may want to escalate the problem to the next support level. A different approach
-is to configure the default notification by email, and escalate the problem via sms
+is to configure the default notification by email, and escalate the problem via SMS
if not already solved.
You can define notification start and end times as additional configuration
Using templates you can share the basic notification attributes such as users or the
`interval` (and override them for the escalation then).
-Using the example from above, you can define additional users being escalated for sms
+Using the example from above, you can define additional users being escalated for SMS
notifications between start and end time.
object User "icinga-oncall-2nd-level" {
vars.mobile = "+1 555 424642"
}
-Define an additional `NotificationCommand` for SMS notifications.
+Define an additional [NotificationCommand](3-monitoring-basics.md#notification-commands) for SMS notifications.
> **Note**
>
"..."
}
-The two new notification escalations are added onto the host `localhost`
+The two new notification escalations are added onto the local host
and its service `ping4` using the `generic-notification` template.
The user `icinga-oncall-2nd-level` will get notified by SMS (`sms-notification`
command) after `30m` until `1h`.
>
> The `interval` was set to 15m in the `generic-notification`
> template example. Lower that value in your escalations by using a secondary
-> template or overriding the attribute directly in the `notifications` array
+> template or by overriding the attribute directly in the `notifications` array
> position for `escalation-sms-2nd-level`.
-If the problem does not get resolved or acknowledged preventing further notifications
+If the problem does not get resolved nor acknowledged preventing further notifications
the `escalation-sms-1st-level` user will be escalated `1h` after the initial problem was
notified, but only for one hour (`2h` as `end` key for the `times` dictionary).
assign where service.name == "ping4"
}
-### <a id="first-notification-delay"></a> First Notification Delay
+### <a id="notification-delay"></a> Notification Delay
-Sometimes the problem in question should not be notified when the first notification
-happens, but a defined time duration afterwards. In Icinga 2 you can use the `times`
-dictionary and set `begin = 15m` as key and value if you want to suppress notifications
-in the first 15 minutes. Leave out the `end` key - if not set, Icinga 2 will not check against any
-end time for this notification.
+Sometimes the problem in question should not be notified when the notification is due
+(the object reaching the `HARD` state) but a defined time duration afterwards. In Icinga 2
+you can use the `times` dictionary and set `begin = 15m` as key and value if you want to
+postpone the notification window for 15 minutes. Leave out the `end` key - if not set,
+Icinga 2 will not check against any end time for this notification. Make sure to
+specify a relatively low notification `interval` to get notified soon enough again.
apply Notification "mail" to Service {
import "generic-notification"
command = "mail-notification"
users = [ "icingaadmin" ]
- times.begin = 15m // delay first notification
+ interval = 5m
+
+ times.begin = 15m // delay notification window
+
+ assign where service.name == "ping4"
+ }
+
+### <a id="disable-renotification"></a> Disable Re-notifications
+
+If you prefer to be notified only once, you can disable re-notifications by setting the
+`interval` attribute to `0`.
+
+ apply Notification "notify-once" to Service {
+ import "generic-notification"
+
+ command = "mail-notification"
+ users = [ "icingaadmin" ]
+
+ interval = 0 // disable re-notification
assign where service.name == "ping4"
}
}
If you are familiar with Icinga 1.x `notification_options` please note that they have been split
-into type and state, and allow more fine granular filtering for example on downtimes and flapping.
+into type and state to allow more fine granular filtering for example on downtimes and flapping.
You can filter for acknowledgements and custom notifications too.
-## <a id="downtimes"></a> Downtimes
+## <a id="commands"></a> Commands
-Downtimes can be scheduled for planned server maintenance or
-any other targetted service outage you are aware of in advance.
+Icinga 2 uses three different command object types to specify how
+checks should be performed, notifications should be sent, and
+events should be handled.
-Downtimes will suppress any notifications, and may trigger other
-downtimes too. If the downtime was set by accident, or the duration
-exceeds the maintenance, you can manually cancel the downtime.
-Planned downtimes will also be taken into account for SLA reporting
-tools calculating the SLAs based on the state and downtime history.
+### <a id="check-commands"></a> Check Commands
-Downtimes may overlap with their start and end times. If there
-are multiple downtimes triggered for one object, the overall downtime depth
-will be more than `1`. This is useful when you want to extend
-your maintenance window taking longer than expected.
+[CheckCommand](6-object-types.md#objecttype-checkcommand) objects define the command line how
+a check is called.
-### <a id="fixed-flexible-downtimes"></a> Fixed and Flexible Downtimes
+[CheckCommand](6-object-types.md#objecttype-checkcommand) objects are referenced by
+[Host](6-object-types.md#objecttype-host) and [Service](6-object-types.md#objecttype-service) objects
+using the `check_command` attribute.
-A `fixed` downtime will be activated at the defined start time, and
-removed at the end time. During this time window the service state
-will change to `NOT-OK` and then actually trigger the downtime.
-Notifications are suppressed and the downtime depth is incremented.
+> **Note**
+>
+> Make sure that the [checker](8-cli-commands.md#features) feature is enabled in order to
+> execute checks.
-Common scenarios are a planned distribution upgrade on your linux
-servers, or database updates in your warehouse. The customer knows
-about a fixed downtime window between 23:00 and 24:00. After 24:00
-all problems should be alerted again. Solution is simple -
-schedule a `fixed` downtime starting at 23:00 and ending at 24:00.
+#### <a id="command-plugin-integration"></a> Integrate the Plugin with a CheckCommand Definition
-Unlike a `fixed` downtime, a `flexible` downtime end does not necessarily
-happen at the provided end time. Instead the downtime will be triggered
-by the state change in the time span defined by start and end time, but
-then last a defined duration in minutes.
+[CheckCommand](6-object-types.md#objecttype-checkcommand) objects require the [ITL template](7-icinga-template-library.md#itl-plugin-check-command)
+`plugin-check-command` to support native plugin based check methods.
-Imagine the following scenario: Your service is frequently polled
-by users trying to grab free deleted domains for immediate registration.
-Between 07:30 and 08:00 the impact will hit for 15 minutes and generate
-a network outage visible to the monitoring. The service is still alive,
-but answering too slow to Icinga 2 service checks.
-For that reason, you may want to schedule a downtime between 07:30 and
-08:00 with a duration of 15 minutes. The downtime will then last from
-its trigger time until the duration is over. After that, the downtime
-is removed (may happen before or after the actual end time!).
+Unless you have done so already, download your check plugin and put it
+into the [PluginDir](4-configuring-icinga-2.md#constants-conf) directory. The following example uses the
+`check_mysql` plugin contained in the Monitoring Plugins package.
-### <a id="scheduling-downtime"></a> Scheduling a downtime
+The plugin path and all command arguments are made a list of
+double-quoted string arguments for proper shell escaping.
-This can either happen through a web interface or by sending an [external command](#external-commands)
-to the external command pipe provided by the `ExternalCommandListener` configuration.
+Call the `check_disk` plugin with the `--help` parameter to see
+all available options. Our example defines warning (`-w`) and
+critical (`-c`) thresholds for the disk usage. Without any
+partition defined (`-p`) it will check all local partitions.
-Fixed downtimes require a start and end time (a duration will be ignored).
-Flexible downtimes need a start and end time for the time span, and a duration
-independent from that time span.
+ icinga@icinga2 $ /usr/lib64/nagios/plugins/check_mysql --help
+ ...
-### <a id="triggered-downtimes"></a> Triggered Downtimes
+ This program tests connections to a MySQL server
-This is optional when scheduling a downtime. If there is already a downtime
-scheduled for a future maintenance, the current downtime can be triggered by
-that downtime. This renders useful if you have scheduled a host downtime and
-are now scheduling a child host's downtime getting triggered by the parent
-downtime on NOT-OK state change.
+ Usage:
+ check_mysql [-d database] [-H host] [-P port] [-s socket]
+ [-u user] [-p password] [-S] [-l] [-a cert] [-k key]
+ [-C ca-cert] [-D ca-dir] [-L ciphers] [-f optfile] [-g group]
-### <a id="recurring-downtimes"></a> Recurring Downtimes
+Next step is to understand how [command parameters](3-monitoring-basics.md#command-passing-parameters)
+are being passed from a host or service object, and add a [CheckCommand](6-object-types.md#objecttype-checkcommand)
+definition based on these required parameters and/or default values.
-[ScheduledDowntime objects](#objecttype-scheduleddowntime) can be used to set up
-recurring downtimes for services.
+Please continue reading in the [plugins section](13-addons-plugins.md#plugins) for additional integration examples.
-Example:
+#### <a id="command-passing-parameters"></a> Passing Check Command Parameters from Host or Service
- apply ScheduledDowntime "backup-downtime" to Service {
- author = "icingaadmin"
- comment = "Scheduled downtime for backup"
+Check command parameters are defined as custom attributes which can be accessed as runtime macros
+by the executed check command.
- ranges = {
- monday = "02:00-03:00"
- tuesday = "02:00-03:00"
- wednesday = "02:00-03:00"
- thursday = "02:00-03:00"
- friday = "02:00-03:00"
- saturday = "02:00-03:00"
- sunday = "02:00-03:00"
- }
+The check command parameters for ITL provided plugin check command definitions are documented
+[here](7-icinga-template-library.md#plugin-check-commands), for example
+[disk](7-icinga-template-library.md#plugin-check-command-disk).
- assign where "backup" in service.groups
- }
+In order to practice passing command parameters you should [integrate your own plugin](3-monitoring-basics.md#command-plugin-integration).
+The following example will use `check_mysql` provided by the [Monitoring Plugins installation](2-getting-started.md#setting-up-check-plugins).
-## <a id="comments"></a> Comments
+Define the default check command custom attributes, for example `mysql_user` and `mysql_password`
+(freely definable naming schema) and optional their default threshold values. You can
+then use these custom attributes as runtime macros for [command arguments](3-monitoring-basics.md#command-arguments)
+on the command line.
-Comments can be added at runtime and are persistent over restarts. You can
-add useful information for others on repeating incidents (for example
-"last time syslog at 100% cpu on 17.10.2013 due to stale nfs mount") which
-is primarly accessible using web interfaces.
+> **Tip**
+>
+> Use a common command type as prefix for your command arguments to increase
+> readability. `mysql_user` helps understanding the context better than just
+> `user` as argument.
-Adding and deleting comment actions are possible through the external command pipe
-provided with the `ExternalCommandListener` configuration. The caller must
-pass the comment id in case of manipulating an existing comment.
+The default custom attributes can be overridden by the custom attributes
+defined in the host or service using the check command `my-mysql`. The custom attributes
+can also be inherited from a parent template using additive inheritance (`+=`).
+ # vim /etc/icinga2/conf.d/commands.conf
-## <a id="acknowledgements"></a> Acknowledgements
+ object CheckCommand "my-mysql" {
+ import "plugin-check-command"
-If a problem is alerted and notified you may signal the other notification
-receipients that you are aware of the problem and will handle it.
+ command = [ PluginDir + "/check_mysql" ] //constants.conf -> const PluginDir
-By sending an acknowledgement to Icinga 2 (using the external command pipe
-provided with `ExternalCommandListener` configuration) all future notifications
-are suppressed, a new comment is added with the provided description and
-a notification with the type `NotificationFilterAcknowledgement` is sent
-to all notified users.
+ arguments = {
+ "-H" = "$mysql_host$"
+ "-u" = {
+ required = true
+ value = "$mysql_user$"
+ }
+ "-p" = "$mysql_password$"
+ "-P" = "$mysql_port$"
+ "-s" = "$mysql_socket$"
+ "-a" = "$mysql_cert$"
+ "-d" = "$mysql_database$"
+ "-k" = "$mysql_key$"
+ "-C" = "$mysql_ca_cert$"
+ "-D" = "$mysql_ca_dir$"
+ "-L" = "$mysql_ciphers$"
+ "-f" = "$mysql_optfile$"
+ "-g" = "$mysql_group$"
+ "-S" = {
+ set_if = "$mysql_check_slave$"
+ description = "Check if the slave thread is running properly."
+ }
+ "-l" = {
+ set_if = "$mysql_ssl$"
+ description = "Use ssl encryption"
+ }
+ }
-### <a id="expiring-acknowledgements"></a> Expiring Acknowledgements
+ vars.mysql_check_slave = false
+ vars.mysql_ssl = false
+ vars.mysql_host = "$address$"
+ }
-Once a problem is acknowledged it may disappear from your `handled problems`
-dashboard and no-one ever looks at it again since it will suppress
-notifications too.
+The check command definition also sets `mysql_host` to the `$address$` default value. You can override
+this command parameter if for example your MySQL host is not running on the same server's ip address.
-This `fire-and-forget` action is quite common. If you're sure that a
-current problem should be resolved in the future at a defined time,
-you can define an expiration time when acknowledging the problem.
+Make sure pass all required command parameters, such as `mysql_user`, `mysql_password` and `mysql_database`.
+`MysqlUsername` and `MysqlPassword` are specified as [global constants](4-configuring-icinga-2.md#constants-conf)
+in this example.
-Icinga 2 will clear the acknowledgement when expired and start to
-re-notify if the problem persists.
+ # vim /etc/icinga2/conf.d/services.conf
+ apply Service "mysql-icinga-db-health" {
+ import "generic-service"
-## <a id="dependencies"></a> Dependencies
+ check_command = "my-mysql"
-Icinga 2 uses host and service [Dependency](#objecttype-dependency) objects.
-The `parent_host_name` and `parent_service_name` attributes are mandatory for
-service dependencies, `parent_host_name` is required for host dependencies.
+ vars.mysql_user = MysqlUsername
+ vars.mysql_password = MysqlPassword
-A service can depend on a host, and vice versa. A service has an implicit
-dependency (parent) to its host. A host to host dependency acts implicit
-as host parent relation.
-When dependencies are calculated, not only the immediate parent is taken into
-account but all parents are inherited.
+ vars.mysql_database = "icinga"
+ vars.mysql_host = "192.168.33.11"
-A common scenario is the Icinga 2 server behind a router. Checking internet
-access by pinging the Google DNS server `google-dns` is a common method, but
-will fail in case the `dsl-router` host is down. Therefore the example below
-defines a host dependency which acts implicit as parent relation too.
+ assign where match("icinga2*", host.name)
+ ignore where host.vars.no_health_check == true
+ }
-Furthermore the host may be reachable but ping probes are dropped by the
-router's firewall. In case the `dsl-router``ping4` service check fails, all
-further checks for the `ping4` service on host `google-dns` service should
-be suppressed. This is achieved by setting the `disable_checks` attribute to `true`.
- object Host "dsl-router" {
- address = "192.168.1.1"
- }
+Take a different example: The example host configuration in [hosts.conf](4-configuring-icinga-2.md#hosts-conf)
+also applies an `ssh` service check. Your host's ssh port is not the default `22`, but set to `2022`.
+You can pass the command parameter as custom attribute `ssh_port` directly inside the service apply rule
+inside [services.conf](4-configuring-icinga-2.md#services-conf):
- object Host "google-dns" {
- address = "8.8.8.8"
- }
-
- apply Service "ping4" {
+ apply Service "ssh" {
import "generic-service"
- check_command = "ping4"
+ check_command = "ssh"
+ vars.ssh_port = 2022 //custom command parameter
- assign where host.address
+ assign where (host.address || host.address6) && host.vars.os == "Linux"
}
- apply Dependency "internet" to Service {
- parent_host_name = "dsl-router"
- disable_checks = true
+If you prefer this being configured at the host instead of the service, modify the host configuration
+object instead. The runtime macro resolving order is described [here](3-monitoring-basics.md#macro-evaluation-order).
- assign where host.name != "dsl-router"
- }
+ object Host NodeName {
+ ...
+ vars.ssh_port = 2022
+ }
+#### <a id="command-passing-parameters-apply-for"></a> Passing Check Command Parameters Using Apply For
-## <a id="custom-attributes"></a> Custom Attributes
+The host `localhost` with the generated services from the `basic-partitions` dictionary (see
+[apply for](3-monitoring-basics.md#using-apply-for) for details) checks a basic set of disk partitions
+with modified custom attributes (warning thresholds at `10%`, critical thresholds at `5%`
+free disk space).
-### <a id="runtime-custom-attributes"></a> Using Custom Attributes at Runtime
+The custom attribute `disk_partition` can either hold a single string or an array of
+string values for passing multiple partitions to the `check_disk` check plugin.
-Custom attributes may be used in command definitions to dynamically change how the command
-is executed.
+ object Host "my-server" {
+ import "generic-host"
+ address = "127.0.0.1"
+ address6 = "::1"
-Additionally there are Icinga 2 features such as the `PerfDataWriter` type
-which use custom attributes to format their output.
+ vars.local_disks["basic-partitions"] = {
+ disk_partitions = [ "/", "/tmp", "/var", "/home" ]
+ }
+ }
-> **Tip**
->
-> Custom attributes are identified by the 'vars' dictionary attribute as short name.
-> Accessing the different attribute keys is possible using the '.' accessor.
+ apply Service for (disk => config in host.vars.local_disks) {
+ import "generic-service"
+ check_command = "my-disk"
-Custom attributes in command definitions or performance data templates are evaluated at
-runtime when executing a command. These custom attributes cannot be used elsewhere
-(e.g. in other configuration attributes).
+ vars += config
-Here is an example of a command definition which uses user-defined custom attributes:
+ vars.disk_wfree = "10%"
+ vars.disk_cfree = "5%"
+ }
- object CheckCommand "my-ping" {
- import "plugin-check-command"
- command = [
- PluginDir + "/check_ping",
- "-4",
- "-H", "$address$",
- "-w", "$ping_wrta$,$ping_wpl$%",
- "-c", "$ping_crta$,$ping_cpl$%",
- "-p", "$ping_packets$",
- "-t", "$ping_timeout$"
- ]
+More details on using arrays in custom attributes can be found in
+[this chapter](3-monitoring-basics.md#custom-attributes).
- vars.ping_wrta = 100
- vars.ping_wpl = 5
- vars.ping_crta = 200
- vars.ping_cpl = 15
- vars.ping_packets = 5
- vars.ping_timeout = 0
- }
-Custom attribute names used at runtime must be enclosed in two `$` signs, e.g.
-`$address$`. When using the `$` sign as single character, you need to escape
-it with an additional dollar sign (`$$`).
+#### <a id="command-arguments"></a> Command Arguments
-### <a id="runtime-custom-attributes-evaluation-order"></a> Runtime Custom Attributes Evaluation Order
+By defining a check command line using the `command` attribute Icinga 2
+will resolve all macros in the static string or array. Sometimes it is
+required to extend the arguments list based on a met condition evaluated
+at command execution. Or making arguments optional - only set if the
+macro value can be resolved by Icinga 2.
-When executing commands Icinga 2 checks the following objects in this order to look
-up custom attributes and their respective values:
+ object CheckCommand "check_http" {
+ import "plugin-check-command"
-1. User object (only for notifications)
-2. Service object
-3. Host object
-4. Command object
-5. Global custom attributes in the Vars constant
+ command = [ PluginDir + "/check_http" ]
-This execution order allows you to define default values for custom attributes
-in your command objects. The `my-ping` command shown above uses this to set
-default values for some of the latency thresholds and timeouts.
+ arguments = {
+ "-H" = "$http_vhost$"
+ "-I" = "$http_address$"
+ "-u" = "$http_uri$"
+ "-p" = "$http_port$"
+ "-S" = {
+ set_if = "$http_ssl$"
+ }
+ "--sni" = {
+ set_if = "$http_sni$"
+ }
+ "-a" = {
+ value = "$http_auth_pair$"
+ description = "Username:password on sites with basic authentication"
+ }
+ "--no-body" = {
+ set_if = "$http_ignore_body$"
+ }
+ "-r" = "$http_expect_body_regex$"
+ "-w" = "$http_warn_time$"
+ "-c" = "$http_critical_time$"
+ "-e" = "$http_expect$"
+ }
+
+ vars.http_address = "$address$"
+ vars.http_ssl = false
+ vars.http_sni = false
+ }
-When using the `my-ping` command you can override all or some of the custom
-attributes in the service definition like this:
+The example shows the `check_http` check command defining the most common
+arguments. Each of them is optional by default and will be omitted if
+the value is not set. For example if the service calling the check command
+does not have `vars.http_port` set, it won't get added to the command
+line.
- object Service "ping" {
- host_name = "localhost"
- check_command = "my-ping"
+If the `vars.http_ssl` custom attribute is set in the service, host or command
+object definition, Icinga 2 will add the `-S` argument based on the `set_if`
+numeric value to the command line. String values are not supported.
- vars.ping_packets = 10 // Overrides the default value of 5 given in the command
- }
+If the macro value cannot be resolved, Icinga 2 will not add the defined argument
+to the final command argument array. Empty strings for macro values won't omit
+the argument.
-If a custom attribute isn't defined anywhere an empty value is used and a warning is
-emitted to the Icinga 2 log.
+That way you can use the `check_http` command definition for both, with and
+without SSL enabled checks saving you duplicated command definitions.
+
+Details on all available options can be found in the
+[CheckCommand object definition](6-object-types.md#objecttype-checkcommand).
-> **Best Practice**
->
-> By convention every host should have an `address` attribute. Hosts
-> which have an IPv6 address should also have an `address6` attribute.
-### <a id="runtime-custom-attribute-env-vars"></a> Runtime Custom Attributes as Environment Variables
+#### <a id="command-environment-variables"></a> Environment Variables
The `env` command object attribute specifies a list of environment variables with values calculated
from either runtime macros or custom attributes which should be exported as environment variables
when passing credentials to database checks:
object CheckCommand "mysql-health" {
- import "plugin-check-command",
+ import "plugin-check-command"
+
+ command = [
+ PluginDir + "/check_mysql"
+ ]
- command = PluginDir + "/check_mysql -H $address$ -d $db$",
+ arguments = {
+ "-H" = "$mysql_address$"
+ "-d" = "$mysql_database$"
+ }
- vars.mysql_user = "icinga_check",
+ vars.mysql_address = "$address$"
+ vars.mysql_database = "icinga"
+ vars.mysql_user = "icinga_check"
vars.mysql_pass = "password"
- env.MYSQLUSER = "$mysql_user$",
+ env.MYSQLUSER = "$mysql_user$"
env.MYSQLPASS = "$mysql_pass$"
}
-### <a id="modified-attributes"></a> Modified Attributes
-Icinga 2 allows you to modify defined object attributes at runtime different to
-the local configuration object attributes. These modified attributes are
-stored as bit-shifted-value and made available in backends. Icinga 2 stores
-modified attributes in its state file and restores them on restart.
-Modified Attributes can be reset using external commands.
+### <a id="notification-commands"></a> Notification Commands
+[NotificationCommand](6-object-types.md#objecttype-notificationcommand) objects define how notifications are delivered to external
+interfaces (E-Mail, XMPP, IRC, Twitter, etc).
-## <a id="runtime-macros"></a> Runtime Macros
+[NotificationCommand](6-object-types.md#objecttype-notificationcommand) objects are referenced by
+[Notification](6-object-types.md#objecttype-notification) objects using the `command` attribute.
-Next to custom attributes there are additional runtime macros made available by Icinga 2.
-These runtime macros reflect the current object state and may change over time while
-custom attributes are configured statically (but can be modified at runtime using
-external commands).
+`NotificationCommand` objects require the [ITL template](7-icinga-template-library.md#itl-plugin-notification-command)
+`plugin-notification-command` to support native plugin-based notifications.
-### <a id="host-runtime-macros"></a> Host Runtime Macros
+> **Note**
+>
+> Make sure that the [notification](8-cli-commands.md#features) feature is enabled
+> in order to execute notification commands.
-The following host custom attributes are available in all commands that are executed for
-hosts or services:
+Below is an example using runtime macros from Icinga 2 (such as `$service.output$` for
+the current check output) sending an email to the user(s) associated with the
+notification itself (`$user.email$`).
- Name | Description
- -----------------------------|--------------
- host.name | The name of the host object.
- host.display_name | The value of the `display_name` attribute.
- host.state | The host's current state. Can be one of `UNREACHABLE`, `UP` and `DOWN`.
- host.state_id | The host's current state. Can be one of `0` (up), `1` (down) and `2` (unreachable).
- host.state_type | The host's current state type. Can be one of `SOFT` and `HARD`.
- host.check_attempt | The current check attempt number.
- host.max_check_attempts | The maximum number of checks which are executed before changing to a hard state.
- host.last_state | The host's previous state. Can be one of `UNREACHABLE`, `UP` and `DOWN`.
- host.last_state_id | The host's previous state. Can be one of `0` (up), `1` (down) and `2` (unreachable).
- host.last_state_type | The host's previous state type. Can be one of `SOFT` and `HARD`.
- host.last_state_change | The last state change's timestamp.
- host.duration_sec | The time since the last state change.
- host.latency | The host's check latency.
- host.execution_time | The host's check execution time.
- host.output | The last check's output.
- host.perfdata | The last check's performance data.
- host.last_check | The timestamp when the last check was executed.
- host.total_services | Number of services associated with the host.
- host.total_services_ok | Number of services associated with the host which are in an `OK` state.
- host.total_services_warning | Number of services associated with the host which are in a `WARNING` state.
- host.total_services_unknown | Number of services associated with the host which are in an `UNKNOWN` state.
- host.total_services_critical | Number of services associated with the host which are in a `CRITICAL` state.
+If you want to specify default values for some of the custom attribute definitions,
+you can add a `vars` dictionary as shown for the `CheckCommand` object.
-### <a id="service-runtime-macros"></a> Service Runtime Macros
+ object NotificationCommand "mail-service-notification" {
+ import "plugin-notification-command"
-The following service macros are available in all commands that are executed for
-services:
+ command = [ SysconfDir + "/icinga2/scripts/mail-notification.sh" ]
- Name | Description
- ---------------------------|--------------
- service.name | The short name of the service object.
- service.display_name | The value of the `display_name` attribute.
- service.check_command | The short name of the command along with any arguments to be used for the check.
- service.state | The service's current state. Can be one of `OK`, `WARNING`, `CRITICAL` and `UNKNOWN`.
- service.state_id | The service's current state. Can be one of `0` (ok), `1` (warning), `2` (critical) and `3` (unknown).
- service.state_type | The service's current state type. Can be one of `SOFT` and `HARD`.
- service.check_attempt | The current check attempt number.
- service.max_check_attempts | The maximum number of checks which are executed before changing to a hard state.
- service.last_state | The service's previous state. Can be one of `OK`, `WARNING`, `CRITICAL` and `UNKNOWN`.
- service.last_state_id | The service's previous state. Can be one of `0` (ok), `1` (warning), `2` (critical) and `3` (unknown).
- service.last_state_type | The service's previous state type. Can be one of `SOFT` and `HARD`.
- service.last_state_change | The last state change's timestamp.
- service.duration_sec | The time since the last state change.
- service.latency | The service's check latency.
- service.execution_time | The service's check execution time.
- service.output | The last check's output.
- service.perfdata | The last check's performance data.
- service.last_check | The timestamp when the last check was executed.
+ env = {
+ NOTIFICATIONTYPE = "$notification.type$"
+ SERVICEDESC = "$service.name$"
+ HOSTALIAS = "$host.display_name$"
+ HOSTADDRESS = "$address$"
+ SERVICESTATE = "$service.state$"
+ LONGDATETIME = "$icinga.long_date_time$"
+ SERVICEOUTPUT = "$service.output$"
+ NOTIFICATIONAUTHORNAME = "$notification.author$"
+ NOTIFICATIONCOMMENT = "$notification.comment$"
+ HOSTDISPLAYNAME = "$host.display_name$"
+ SERVICEDISPLAYNAME = "$service.display_name$"
+ USEREMAIL = "$user.email$"
+ }
+ }
-### <a id="command-runtime-macros"></a> Command Runtime Macros
+The command attribute in the `mail-service-notification` command refers to the following
+shell script. The macros specified in the `env` array are exported
+as environment variables and can be used in the notification script:
-The following custom attributes are available in all commands:
+ #!/usr/bin/env bash
+ template=$(cat <<TEMPLATE
+ ***** Icinga *****
- Name | Description
- -----------------------|--------------
- command.name | The name of the command object.
+ Notification Type: $NOTIFICATIONTYPE
-### <a id="user-runtime-macros"></a> User Runtime Macros
+ Service: $SERVICEDESC
+ Host: $HOSTALIAS
+ Address: $HOSTADDRESS
+ State: $SERVICESTATE
-The following custom attributes are available in all commands that are executed for
-users:
+ Date/Time: $LONGDATETIME
- Name | Description
- -----------------------|--------------
- user.name | The name of the user object.
- user.display_name | The value of the display_name attribute.
+ Additional Info: $SERVICEOUTPUT
-### <a id="notification-runtime-macros"></a> Notification Runtime Macros
+ Comment: [$NOTIFICATIONAUTHORNAME] $NOTIFICATIONCOMMENT
+ TEMPLATE
+ )
- Name | Description
- -----------------------|--------------
- notification.type | The type of the notification.
- notification.author | The author of the notification comment, if existing.
- notification.comment | The comment of the notification, if existing.
+ /usr/bin/printf "%b" $template | mail -s "$NOTIFICATIONTYPE - $HOSTDISPLAYNAME - $SERVICEDISPLAYNAME is $SERVICESTATE" $USEREMAIL
-### <a id="global-runtime-macros"></a> Global Runtime Macros
+> **Note**
+>
+> This example is for `exim` only. Requires changes for `sendmail` and
+> other MTAs.
-The following macros are available in all executed commands:
+While it's possible to specify the entire notification command right
+in the NotificationCommand object it is generally advisable to create a
+shell script in the `/etc/icinga2/scripts` directory and have the
+NotificationCommand object refer to that.
- Name | Description
- -----------------------|--------------
- icinga.timet | Current UNIX timestamp.
- icinga.long_date_time | Current date and time including timezone information. Example: `2014-01-03 11:23:08 +0000`
- icinga.short_date_time | Current date and time. Example: `2014-01-03 11:23:08`
- icinga.date | Current date. Example: `2014-01-03`
- icinga.time | Current time including timezone information. Example: `11:23:08 +0000`
- icinga.uptime | Current uptime of the Icinga 2 process.
+### <a id="event-commands"></a> Event Commands
-The following macros provide global statistics:
+Unlike notifications, event commands for hosts/services are called on every
+check execution if one of these conditions match:
- Name | Description
- ----------------------------------|--------------
- icinga.num_services_ok | Current number of services in state 'OK'.
- icinga.num_services_warning | Current number of services in state 'Warning'.
- icinga.num_services_critical | Current number of services in state 'Critical'.
- icinga.num_services_unknown | Current number of services in state 'Unknown'.
- icinga.num_services_pending | Current number of pending services.
- icinga.num_services_unreachable | Current number of unreachable services.
- icinga.num_services_flapping | Current number of flapping services.
- icinga.num_services_in_downtime | Current number of services in downtime.
- icinga.num_services_acknowledged | Current number of acknowledged service problems.
- icinga.num_hosts_up | Current number of hosts in state 'Up'.
- icinga.num_hosts_down | Current number of hosts in state 'Down'.
- icinga.num_hosts_unreachable | Current number of unreachable hosts.
- icinga.num_hosts_flapping | Current number of flapping hosts.
- icinga.num_hosts_in_downtime | Current number of hosts in downtime.
- icinga.num_hosts_acknowledged | Current number of acknowledged host problems.
+* The host/service is in a [soft state](3-monitoring-basics.md#hard-soft-states)
+* The host/service state changes into a [hard state](3-monitoring-basics.md#hard-soft-states)
+* The host/service state recovers from a [soft or hard state](3-monitoring-basics.md#hard-soft-states) to [OK](3-monitoring-basics.md#service-states)/[Up](3-monitoring-basics.md#host-states)
+[EventCommand](6-object-types.md#objecttype-eventcommand) objects are referenced by
+[Host](6-object-types.md#objecttype-host) and [Service](6-object-types.md#objecttype-service) objects
+using the `event_command` attribute.
-## <a id="check-result-freshness"></a> Check Result Freshness
+Therefore the `EventCommand` object should define a command line
+evaluating the current service state and other service runtime attributes
+available through runtime vars. Runtime macros such as `$service.state_type$`
+and `$service.state$` will be processed by Icinga 2 helping on fine-granular
+events being triggered.
-In Icinga 2 active check freshness is enabled by default. It is determined by the
-`check_interval` attribute and no incoming check results in that period of time.
+Common use case scenarios are a failing HTTP check requiring an immediate
+restart via event command, or if an application is locked and requires
+a restart upon detection.
- threshold = last check execution time + check interval
+`EventCommand` objects require the ITL template `plugin-event-command`
+to support native plugin based checks.
-Passive check freshness is calculated from the `check_interval` attribute if set.
+#### <a id="event-command-restart-service-daemon"></a> Use Event Commands to Restart Service Daemon
- threshold = last check result time + check interval
+The following example will triggert a restart of the `httpd` daemon
+via ssh when the `http` service check fails. If the service state is
+`OK`, it will not trigger any event action.
-If the freshness checks are invalid, a new check is executed defined by the
-`check_command` attribute.
+Requirements:
+* ssh connection
+* icinga user with public key authentication
+* icinga user with sudo permissions for restarting the httpd daemon.
-## <a id="check-flapping"></a> Check Flapping
+Example on Debian:
-The flapping algorithm used in Icinga 2 does not store the past states but
-calculcates the flapping threshold from a single value based on counters and
-half-life values. Icinga 2 compares the value with a single flapping threshold
-configuration attribute named `flapping_threshold`.
+ # ls /home/icinga/.ssh/
+ authorized_keys
-Flapping detection can be enabled or disabled using the `enable_flapping` attribute.
+ # visudo
+ icinga ALL=(ALL) NOPASSWD: /etc/init.d/apache2 restart
-## <a id="volatile-services"></a> Volatile Services
+Define a generic [EventCommand](6-object-types.md#objecttype-eventcommand) object `event_by_ssh`
+which can be used for all event commands triggered using ssh:
-By default all services remain in a non-volatile state. When a problem
-occurs, the `SOFT` state applies and once `max_check_attempts` attribute
-is reached with the check counter, a `HARD` state transition happens.
-Notifications are only triggered by `HARD` state changes and are then
-re-sent defined by the `interval` attribute.
+ /* pass event commands through ssh */
+ object EventCommand "event_by_ssh" {
+ import "plugin-event-command"
-It may be reasonable to have a volatile service which stays in a `HARD`
-state type if the service stays in a `NOT-OK` state. That way each
-service recheck will automatically trigger a notification unless the
-service is acknowledged or in a scheduled downtime.
+ command = [ PluginDir + "/check_by_ssh" ]
+ arguments = {
+ "-H" = "$event_by_ssh_address$"
+ "-p" = "$event_by_ssh_port$"
+ "-C" = "$event_by_ssh_command$"
+ "-l" = "$event_by_ssh_logname$"
+ "-i" = "$event_by_ssh_identity$"
+ "-q" = {
+ set_if = "$event_by_ssh_quiet$"
+ }
+ "-w" = "$event_by_ssh_warn$"
+ "-c" = "$event_by_ssh_crit$"
+ "-t" = "$event_by_ssh_timeout$"
+ }
-## <a id="external-commands"></a> External Commands
+ vars.event_by_ssh_address = "$address$"
+ vars.event_by_ssh_quiet = false
+ }
-Icinga 2 provides an external command pipe for processing commands
-triggering specific actions (for example rescheduling a service check
-through the web interface).
+The actual event command only passes the `event_by_ssh_command` attribute.
+The `event_by_ssh_service` custom attribute takes care of passing the correct
+daemon name, while `test $service.state_id$ -gt 0` makes sure that the daemon
+is only restarted when the service is not in an `OK` state.
-In order to enable the `ExternalCommandListener` configuration use the
-following command and restart Icinga 2 afterwards:
- # icinga2-enable-feature command
+ object EventCommand "event_by_ssh_restart_service" {
+ import "event_by_ssh"
-Icinga 2 creates the command pipe file as `/var/run/icinga2/cmd/icinga2.cmd`
-using the default configuration.
+ //only restart the daemon if state > 0 (not-ok)
+ //requires sudo permissions for the icinga user
+ vars.event_by_ssh_command = "test $service.state_id$ -gt 0 && sudo /etc/init.d/$event_by_ssh_service$ restart"
+ }
-Web interfaces and other Icinga addons are able to send commands to
-Icinga 2 through the external command pipe, for example for rescheduling
-a forced service check:
- # /bin/echo "[`date +%s`] SCHEDULE_FORCED_SVC_CHECK;localhost;ping4;`date +%s`" >> /var/run/icinga2/cmd/icinga2.cmd
+Now set the `event_command` attribute to `event_by_ssh_restart_service` and tell it
+which service should be restarted using the `event_by_ssh_service` attribute.
- # tail -f /var/log/messages
+ object Service "http" {
+ import "generic-service"
+ host_name = "remote-http-host"
+ check_command = "http"
- Oct 17 15:01:25 icinga-server icinga2: Executing external command: [1382014885] SCHEDULE_FORCED_SVC_CHECK;localhost;ping4;1382014885
- Oct 17 15:01:25 icinga-server icinga2: Rescheduling next check for service 'ping4'
+ event_command = "event_by_ssh_restart_service"
+ vars.event_by_ssh_service = "$host.vars.httpd_name$"
-By default the command pipe file is owned by the group `icingacmd` with read/write
-permissions. Add your webserver's user to the group `icingacmd` to
-enable sending commands to Icinga 2 through your web interface:
+ //vars.event_by_ssh_logname = "icinga"
+ //vars.event_by_ssh_identity = "/home/icinga/.ssh/id_rsa.pub"
+ }
- # usermod -G -a icingacmd www-data
-Debian packages use `nagios` as the default user and group name. Therefore change `icingacmd` to
-`nagios`.
+Each host with this service then must define the `httpd_name` custom attribute
+(for example generated from your cmdb):
-### <a id="external-command-list"></a> External Command List
+ object Host "remote-http-host" {
+ import "generic-host"
+ address = "192.168.1.100"
-A list of currently supported external commands can be found [here](#external-commands-list-detail)
+ vars.httpd_name = "apache2"
+ }
-Detailed information on the commands and their required parameters can be found
-on the [Icinga 1.x documentation](http://docs.icinga.org/latest/en/extcommands2.html).
+You can testdrive this example by manually stopping the `httpd` daemon
+on your `remote-http-host`. Enable the `debuglog` feature and tail the
+`/var/log/icinga2/debug.log` file.
+Remote Host Terminal:
-## <a id="event-handlers"></a> Event Handlers
+ # date; service apache2 status
+ Mon Sep 15 18:57:39 CEST 2014
+ Apache2 is running (pid 23651).
+ # date; service apache2 stop
+ Mon Sep 15 18:57:47 CEST 2014
+ [ ok ] Stopping web server: apache2 ... waiting .
-Event handlers are defined as `EventCommand` objects in Icinga 2.
+Icinga 2 Host Terminal:
-Unlike notifications event commands are called on every host/service execution
-if defined. Therefore the `EventCommand` object should define a command line
-evaluating the current service state and other service runtime attributes
-available through runtime macros. Runtime macros such as `$service.state_type$`
-and `$service.state$` will be processed by Icinga 2 helping on fine-granular
-events being triggered.
+ [2014-09-15 18:58:32 +0200] notice/Process: Running command '/usr/lib64/nagios/plugins/check_http' '-I' '192.168.1.100': PID 32622
+ [2014-09-15 18:58:32 +0200] notice/Process: PID 32622 ('/usr/lib64/nagios/plugins/check_http' '-I' '192.168.1.100') terminated with exit code 2
+ [2014-09-15 18:58:32 +0200] notice/Checkable: State Change: Checkable remote-http-host!http soft state change from OK to CRITICAL detected.
+ [2014-09-15 18:58:32 +0200] notice/Checkable: Executing event handler 'event_by_ssh_restart_service' for service 'remote-http-host!http'
+ [2014-09-15 18:58:32 +0200] notice/Process: Running command '/usr/lib64/nagios/plugins/check_by_ssh' '-C' 'test 2 -gt 0 && sudo /etc/init.d/apache2 restart' '-H' '192.168.1.100': PID 32623
+ [2014-09-15 18:58:33 +0200] notice/Process: PID 32623 ('/usr/lib64/nagios/plugins/check_by_ssh' '-C' 'test 2 -gt 0 && sudo /etc/init.d/apache2 restart' '-H' '192.168.1.100') terminated with exit code 0
-Common use case scenarios are a failing HTTP check requiring an immediate
-restart via event command, or if an application is locked and requires
-a restart upon detection.
+Remote Host Terminal:
+ # date; service apache2 status
+ Mon Sep 15 18:58:44 CEST 2014
+ Apache2 is running (pid 24908).
-## <a id="logging"></a> Logging
-Icinga 2 supports three different types of logging:
+## <a id="dependencies"></a> Dependencies
-* File logging
-* Syslog (on *NIX-based operating systems)
-* Console logging (`STDOUT` on tty)
+Icinga 2 uses host and service [Dependency](6-object-types.md#objecttype-dependency) objects
+for determing their network reachability.
-You can enable additional loggers using the `icinga2-enable-feature`
-and `icinga2-disable-feature` commands to configure loggers:
+A service can depend on a host, and vice versa. A service has an implicit
+dependency (parent) to its host. A host to host dependency acts implicitly
+as host parent relation.
+When dependencies are calculated, not only the immediate parent is taken into
+account but all parents are inherited.
-Feature | Description
----------|------------
-debuglog | Debug log (path: `/var/log/icinga2/debug.log`, severity: `debug` or higher)
-mainlog | Main log (path: `/var/log/icinga2/icinga2.log`, severity: `information` or higher)
-syslog | Syslog (severity: `warning` or higher)
+The `parent_host_name` and `parent_service_name` attributes are mandatory for
+service dependencies, `parent_host_name` is required for host dependencies.
+[Apply rules](3-monitoring-basics.md#using-apply) will allow you to
+[determine these attributes](3-monitoring-basics.md#dependencies-apply-custom-attributes) in a more
+dynamic fashion if required.
-By default file the `mainlog` feature is enabled. When running Icinga 2
-on a terminal log messages with severity `information` or higher are
-written to the console.
+ parent_host_name = "core-router"
+ parent_service_name = "uplink-port"
+Notifications are suppressed by default if a host or service becomes unreachable.
+You can control that option by defining the `disable_notifications` attribute.
-## <a id="performance-data"></a> Performance Data
+ disable_notifications = false
-When a host or service check is executed plugins should provide so-called
-`performance data`. Next to that additional check performance data
-can be fetched using Icinga 2 runtime macros such as the check latency
-or the current service state (or additional custom attributes).
+If the dependency should be triggered in the parent object's soft state, you
+need to set `ignore_soft_states` to `false`.
-The performance data can be passed to external applications which aggregate and
-store them in their backends. These tools usually generate graphs for historical
-reporting and trending.
+The dependency state filter must be defined based on the parent object being
+either a host (`Up`, `Down`) or a service (`OK`, `Warning`, `Critical`, `Unknown`).
-Well-known addons processing Icinga performance data are PNP4Nagios,
-inGraph and Graphite.
+The following example will make the dependency fail and trigger it if the parent
+object is **not** in one of these states:
-### <a id="writing-performance-data-files"></a> Writing Performance Data Files
+ states = [ OK, Critical, Unknown ]
-PNP4Nagios, inGraph and Graphios use performance data collector daemons to fetch
-the current performance files for their backend updates.
+Rephrased: If the parent service object changes into the `Warning` state, this
+dependency will fail and render all child objects (hosts or services) unreachable.
-Therefore the Icinga 2 `PerfdataWriter` object allows you to define
-the output template format for host and services backed with Icinga 2
-runtime vars.
+You can determine the child's reachability by querying the `is_reachable` attribute
+in for example [DB IDO](22-appendix.md#schema-db-ido-extensions).
- host_format_template = "DATATYPE::HOSTPERFDATA\tTIMET::$icinga.timet$\tHOSTNAME::$host.name$\tHOSTPERFDATA::$host.perfdata$\tHOSTCHECKCOMMAND::$host.checkcommand$\tHOSTSTATE::$host.state$\tHOSTSTATETYPE::$host.statetype$"
- service_format_template = "DATATYPE::SERVICEPERFDATA\tTIMET::$icinga.timet$\tHOSTNAME::$host.name$\tSERVICEDESC::$service.description$\tSERVICEPERFDATA::$service.perfdata$\tSERVICECHECKCOMMAND::$service.checkcommand$\tHOSTSTATE::$host.state$\tHOSTSTATETYPE::$host.statetype$\tSERVICESTATE::$service.state$\tSERVICESTATETYPE::$service.statetype$"
+### <a id="dependencies-implicit-host-service"></a> Implicit Dependencies for Services on Host
-The default templates are already provided with the Icinga 2 feature configuration
-which can be enabled using
+Icinga 2 automatically adds an implicit dependency for services on their host. That way
+service notifications are suppressed when a host is `DOWN` or `UNREACHABLE`. This dependency
+does not overwrite other dependencies and implicitely sets `disable_notifications = true` and
+`states = [ Up ]` for all service objects.
- # icinga2-enable-feature perfdata
+Service checks are still executed. If you want to prevent them from happening, you can
+apply the following dependency to all services setting their host as `parent_host_name`
+and disabling the checks. `assign where true` matches on all `Service` objects.
-By default all performance data files are rotated in a 15 seconds interval into
-the `/var/spool/icinga2/perfdata/` directory as `host-perfdata.<timestamp>` and
-`service-perfdata.<timestamp>`.
-External collectors need to parse the rotated performance data files and then
-remove the processed files.
+ apply Dependency "disable-host-service-checks" to Service {
+ disable_checks = true
+ assign where true
+ }
-### <a id="graphite-carbon-cache-writer"></a> Graphite Carbon Cache Writer
+### <a id="dependencies-network-reachability"></a> Dependencies for Network Reachability
-While there are some Graphite collector scripts and daemons like Graphios available for
-Icinga 1.x it's more reasonable to directly process the check and plugin performance
-in memory in Icinga 2. Once there are new metrics available, Icinga 2 will directly
-write them to the defined Graphite Carbon daemon tcp socket.
+A common scenario is the Icinga 2 server behind a router. Checking internet
+access by pinging the Google DNS server `google-dns` is a common method, but
+will fail in case the `dsl-router` host is down. Therefore the example below
+defines a host dependency which acts implicitly as parent relation too.
-You can enable the feature using
+Furthermore the host may be reachable but ping probes are dropped by the
+router's firewall. In case the `dsl-router`'s `ping4` service check fails, all
+further checks for the `ping4` service on host `google-dns` service should
+be suppressed. This is achieved by setting the `disable_checks` attribute to `true`.
- # icinga2-enable-feature graphite
+ object Host "dsl-router" {
+ import "generic-host"
+ address = "192.168.1.1"
+ }
-By default the `GraphiteWriter` object expects the Graphite Carbon Cache to listen at
-`127.0.0.1` on port `2003`.
+ object Host "google-dns" {
+ import "generic-host"
+ address = "8.8.8.8"
+ }
-The current naming schema is
+ apply Service "ping4" {
+ import "generic-service"
- icinga.<hostname>.<metricname>
- icinga.<hostname>.<servicename>.<metricname>
+ check_command = "ping4"
+ assign where host.address
+ }
+ apply Dependency "internet" to Host {
+ parent_host_name = "dsl-router"
+ disable_checks = true
+ disable_notifications = true
-## <a id="status-data"></a> Status Data
+ assign where host.name != "dsl-router"
+ }
-Icinga 1.x writes object configuration data and status data in a cyclic
-interval to its `objects.cache` and `status.dat` files. Icinga 2 provides
-the `StatusDataWriter` object which dumps all configuration objects and
-status updates in a regular interval.
+ apply Dependency "internet" to Service {
+ parent_host_name = "dsl-router"
+ parent_service_name = "ping4"
+ disable_checks = true
- # icinga2-enable-feature statusdata
+ assign where host.name != "dsl-router"
+ }
-Icinga 1.x Classic UI requires this data set as part of its backend.
+### <a id="dependencies-apply-custom-attributes"></a> Apply Dependencies based on Custom Attributes
-> **Note**
->
-> If you are not using any web interface or addon which uses these files
-> you can safely disable this feature.
+You can use [apply rules](3-monitoring-basics.md#using-apply) to set parent or
+child attributes e.g. `parent_host_name` to other object's
+attributes.
+A common example are virtual machines hosted on a master. The object
+name of that master is auto-generated from your CMDB or VMWare inventory
+into the host's custom attributes (or a generic template for your
+cloud).
+Define your master host object:
-## <a id="compat-logging"></a> Compat Logging
+ /* your master */
+ object Host "master.example.com" {
+ import "generic-host"
+ }
-The Icinga 1.x log format is considered being the `Compat Log`
-in Icinga 2 provided with the `CompatLogger` object.
+Add a generic template defining all common host attributes:
-These logs are not only used for informational representation in
-external web interfaces parsing the logs, but also to generate
-SLA reports and trends in Icinga 1.x Classic UI. Futhermore the
-`Livestatus` feature uses these logs for answering queries to
-historical tables.
+ /* generic template for your virtual machines */
+ template Host "generic-vm" {
+ import "generic-host"
+ }
-The `CompatLogger` object can be enabled with
+Add a template for all hosts on your example.com cloud setting
+custom attribute `vm_parent` to `master.example.com`:
- # icinga2-enable-feature compatlog
+ template Host "generic-vm-example.com" {
+ import "generic-vm"
+ vars.vm_parent = "master.example.com"
+ }
-By default, the Icinga 1.x log file called `icinga.log` is located
-in `/var/log/icinga2/compat`. Rotated log files are moved into
-`var/log/icinga2/compat/archives`.
+Define your guest hosts:
-The format cannot be changed without breaking compatibility to
-existing log parsers.
+ object Host "www.example1.com" {
+ import "generic-vm-master.example.com"
+ }
- # tail -f /var/log/icinga2/compat/icinga.log
+ object Host "www.example2.com" {
+ import "generic-vm-master.example.com"
+ }
- [1382115688] LOG ROTATION: HOURLY
- [1382115688] LOG VERSION: 2.0
- [1382115688] HOST STATE: CURRENT;localhost;UP;HARD;1;
- [1382115688] SERVICE STATE: CURRENT;localhost;disk;WARNING;HARD;1;
- [1382115688] SERVICE STATE: CURRENT;localhost;http;OK;HARD;1;
- [1382115688] SERVICE STATE: CURRENT;localhost;load;OK;HARD;1;
- [1382115688] SERVICE STATE: CURRENT;localhost;ping4;OK;HARD;1;
- [1382115688] SERVICE STATE: CURRENT;localhost;ping6;OK;HARD;1;
- [1382115688] SERVICE STATE: CURRENT;localhost;processes;WARNING;HARD;1;
- [1382115688] SERVICE STATE: CURRENT;localhost;ssh;OK;HARD;1;
- [1382115688] SERVICE STATE: CURRENT;localhost;users;OK;HARD;1;
- [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;disk;1382115705
- [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;http;1382115705
- [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;load;1382115705
- [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;ping4;1382115705
- [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;ping6;1382115705
- [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;processes;1382115705
- [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;ssh;1382115705
- [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;users;1382115705
- [1382115731] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;localhost;ping6;2;critical test|
- [1382115731] SERVICE ALERT: localhost;ping6;CRITICAL;SOFT;2;critical test
+Apply the host dependency to all child hosts importing the
+`generic-vm` template and set the `parent_host_name`
+to the previously defined custom attribute `host.vars.vm_parent`.
+ apply Dependency "vm-host-to-parent-master" to Host {
+ parent_host_name = host.vars.vm_parent
+ assign where "generic-vm" in host.templates
+ }
+You can extend this example, and make your services depend on the
+`master.example.com` host too. Their local scope allows you to use
+`host.vars.vm_parent` similar to the example above.
+
+ apply Dependency "vm-service-to-parent-master" to Service {
+ parent_host_name = host.vars.vm_parent
+ assign where "generic-vm" in host.templates
+ }
+
+That way you don't need to wait for your guest hosts becoming
+unreachable when the master host goes down. Instead the services
+will detect their reachability immediately when executing checks.
+
+> **Note**
+>
+> This method with setting locally scoped variables only works in
+> apply rules, but not in object definitions.
-## <a id="check-result-files"></a> Check Result Files
-Icinga 1.x writes its check result files into a temporary spool directory
-where it reads these check result files in a regular interval from.
-While this is extremly inefficient in performance regards it has been
-rendered useful for passing passive check results directly into Icinga 1.x
-skipping the external command pipe.
+### <a id="dependencies-agent-checks"></a> Dependencies for Agent Checks
-Several clustered/distributed environments and check-aggregation addons
-use that method. In order to support step-by-step migration of these
-environments, Icinga 2 ships the `CheckResultReader` object.
+Another classic example are agent based checks. You would define a health check
+for the agent daemon responding to your requests, and make all other services
+querying that daemon depend on that health check.
-There is no feature configuration available, but it must be defined
-on-demand in your Icinga 2 objects configuration.
+The following configuration defines two nrpe based service checks `nrpe-load`
+and `nrpe-disk` applied to the `nrpe-server`. The health check is defined as
+`nrpe-health` service.
- object CheckResultReader "reader" {
- spool_dir = "/data/check-results"
+ apply Service "nrpe-health" {
+ import "generic-service"
+ check_command = "nrpe"
+ assign where match("nrpe-*", host.name)
}
+ apply Service "nrpe-load" {
+ import "generic-service"
+ check_command = "nrpe"
+ vars.nrpe_command = "check_load"
+ assign where match("nrpe-*", host.name)
+ }
+ apply Service "nrpe-disk" {
+ import "generic-service"
+ check_command = "nrpe"
+ vars.nrpe_command = "check_disk"
+ assign where match("nrpe-*", host.name)
+ }
+ object Host "nrpe-server" {
+ import "generic-host"
+ address = "192.168.1.5"
+ }
+ apply Dependency "disable-nrpe-checks" to Service {
+ parent_service_name = "nrpe-health"
+ states = [ OK ]
+ disable_checks = true
+ disable_notifications = true
+ assign where service.check_command == "nrpe"
+ ignore where service.name == "nrpe-health"
+ }
+The `disable-nrpe-checks` dependency is applied to all services
+on the `nrpe-service` host using the `nrpe` check_command attribute
+but not the `nrpe-health` service itself.
projects / icinga2 / blobdiff