From: Michael Friedrich Date: Thu, 17 Oct 2013 22:11:35 +0000 (+0200) Subject: Documentation: Add Notifications, Escalations, Filters, Delay. X-Git-Tag: v0.0.3~51 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=dd1ec43cd0c0a40b4d0f47b883a3cffb8dc28cec;p=icinga2 Documentation: Add Notifications, Escalations, Filters, Delay. --- diff --git a/doc/3.02-commands.md b/doc/3.02-commands.md index 36a7ea554..4fb528895 100644 --- a/doc/3.02-commands.md +++ b/doc/3.02-commands.md @@ -4,7 +4,7 @@ Icinga 2 uses three different command object types to specify how checks should be performed, notifications should be sent and events should be handled. -> **Tip** +> **Note** > > Define the `$plugindir$` macro in your global `IcingaMacros` variable > (located in `/etc/icinga2/conf.d/macros.conf` by default) and use @@ -12,6 +12,22 @@ events should be handled. > Put your plugins and scripts into the directory defined by the `$plugindir$` macro > and make sure they are executable by the Icinga 2 user. +### Environment Macros + +If your plugins require environment macros instead of command arguments you have +to define all macros in the `export_macros` attribute as list. + + export_macros = [ + "HOSTNAME", + "SERVICEDESC", + "SERVICESTATE" + ] + +> **Note** +> +> Use templates to define global `export_macros` attributes for the three +> command types and let each command object inherit the attribute. + ### Check Commands `CheckCommand` objects define the command line how a check is called. diff --git a/doc/3.04-notifications.md b/doc/3.04-notifications.md index d8ccae517..c7546f20b 100644 --- a/doc/3.04-notifications.md +++ b/doc/3.04-notifications.md @@ -1,13 +1,299 @@ ## Notifications -TODO - Notifications on alerts are an integral part of your Icinga 2 monitoring application. +There are many ways of getting a notification to the actual receiver - Email, XMPP, +IRC, Twitter, etc. The default method for executing a notification command are +plugin scripts used for notifications. +These may either be shell commands to invoke a system call to the `mail` binary +or your own script fetching available macro values and doing proper formatting +before sending the notification. +Other mechanism will require writing the notification string into an api processing +it there (for example ticket system integration). + +Such notification plugins are available from community users and professionals for +example on the [MonitoringExchange](http://www.monitoringexchange.org) or the +[Icinga Wiki](https://wiki.icinga.org). Or you'll write your own and share it. + +A notification requires one or more users (and/or user groups) who will be notified +in case. These users must have all macro attributes defined which will be used in +the `NotificationCommand` on execution, for example `email` as macro dictionary key +is referenced as `$USEREMAIL$`. + +The user `icingaadmin` in this examples will get notified only on `WARNING` and +`CRITICAL` states and `problem` and `recovery` notification types. + + object User "icingaadmin" { + display_name = "Icinga 2 Admin", + enable_notifications = 1, + notification_state_filter = (StateFilterWarning | + StateFilterCritical), + notification_type_filter = (NotificationFilterProblem | + NotificationFilterRecovery), + macros = { + "email" = "icinga@localhost", + "pager" = "+49123456789" + } + } + +> **Note** +> +> If you don't set the `notification_state_filter` and `notification_type_filter` +> configuration attributes for the `User` object, all states and types will be +> notified. + You should choose which information you (and your notified users) are interested in case of emergency, and also which information does not provide any value to you and your environment. +> **Note** +> +> The chain of attribute inheritance including the (additive) macro dictionary for +> notifications will allow granular macros for every specific use case, such as +> `$mail$` or `$mobile$` as `User` macros available in `NotificationCommand`. + + Service -> Notification -> Command -> User + +There are various macros available at runtime execution of the `NotificationCommand`. +The example below may or may not fit your needs. + + object NotificationCommand "mail-notification" inherits "plugin-notification-command" { + command = [ + "/usr/bin/printf", + "\"%b\"", + {{{\"***** Icinga ***** + + Notification Type: $NOTIFICATIONTYPE$ + + Service: $SERVICEDESC$ + Host: $HOSTALIAS$ + Address: $HOSTADDRESS$ + State: $SERVICESTATE$ + + Date/Time: $LONGDATETIME$ + + Additional Info: $SERVICEOUTPUT$ + + Comment: [$NOTIFICATIONAUTHORNAME$] $NOTIFICATIONCOMMENT$\"}}}, + "/bin/mail", + "-s", + "\"$NOTIFICATIONTYPE$ - $HOSTDISPLAYNAME$ - $SERVICEDISPLAYNAME$ is $SERVICESTATE$\"", + "$USEREMAIL$" + ] + } + +> **Note** +> +> Alternatively you can use the `export_macros` attributes to export all required +> macros into the environment. + +You can add all shared attributes to a `Notification` template which is inherited +to the defined notifications. That way you'll save duplicated attributes in each +`Notification` object. Attributes can be overridden locally. + + template Notification "generic-notification" { + notification_interval = 15m, + + notification_command = "mail-service-notification", + + notification_state_filter = (StateFilterWarning | + StateFilterCritical | + StateFilterUnknown), + notification_type_filter = (NotificationFilterProblem | + NotificationFilterAcknowledgement | + NotificationFilterRecovery | + NotificationFilterCustom | + NotificationFilterFlappingStart | + NotificationFilterFlappingEnd | + NotificationFilterDowntimeStart | + NotificationFilterDowntimeEnd | + NotificationFilterDowntimeRemoved), + + notification_period = "24x7" + } + +> **Note** +> +> The `TimePeriod` `24x7` is shipped as example configuration with Icinga 2. + +Use the `generic-notification` template for the `mail` notification defined +inline to the host's service `ping4` and assign the `icingaadmin` user. + + object Host "localhost" { + services["ping4"] = { + notifications["mail"] = { + templates = [ "generic-notification" ], + notification_command = "mail-notification", + users = [ "icingaadmin" ], + } + } + } + +Notifications can be defined in `Service` templates inherited to the objects. + +> **Note** +> +> Instead of assigning users to notifications, you can also add the `user_groups` +> attribute with a list of user groups to the `Notification` object. Icinga 2 will +> resolve all group members and send notifications to all of them. + ### Notification Escalations -TODO +When a problem notification is sent and a problem still exists after re-notification +you may want to escalate the problem to the next support level. A different approach +is to configure the default notification by email, and escalate the problem via sms +if not already solved. + +You can define notification start and end times as additional configuration +attributes making the `Notification` object a so-called `notification escalation`. +Using templates you can share the basic notification attributes such as users or the +`notification_interval` (and override them for the escalation then). + +Using the example from above, you can define additional users being escalated for sms +notifications between start and end time. + +> **Note** +> +> `notification_state_filter` and `notification_type_filter` configuration attributes +> are not set in this example. + + object User "icinga-oncall-2nd-level" { + display_name = "Icinga 2nd Level", + enable_notifications = 1, + + macros = { + "mobile" = "+49123456781" + } + } + + object User "icinga-oncall-1st-level" { + display_name = "Icinga 1st Level", + enable_notifications = 1, + + macros = { + "mobile" = "+49123456782" + } + } + +Define an additional `NotificationCommand` for sms notifications. + +> **Note** +> +> The example is not complete as there are many different sms providers. +> Please note that sending sms notifications will require an sms provider +> or local hardware with a sim card active. + + object NotificationCommand "sms-notification" { + command = "$plugindir$/send_sms_notification $mobile$ ..." + } + +The two new notification escalations are added onto the host `localhost` +and its service `ping4` using the `generic-notification` template. +The user `icinga-oncall-2nd-level` will get notified by sms (`sms-notification` +command) after `30m` until `1h`. + +> **Note** +> +> The `notification_interval` was set to 15m in the `generic-notification` +> template example. Lower that value in your escalations by using a secondary +> template or overriding the attribute directly in the `notifications` array +> position for `escalation-sms-2nd-level`. + +If the problem does not get resolved or acknowledged preventing further notifications +the `escalation-sms-1st-level` user will be escalated `1h` after the initial problem was +notified, but only for one hour (`2h` as `end` key for the `times` dictionary). + + object Host "localhost" { + services["ping4"] = { + notifications["mail"] = { + templates = [ "generic-notification" ], + notification_command = "mail-notification", + users = [ "icingaadmin" ], + }, + notifications["escalation-sms-2nd-level"] = { + templates = [ "generic-notification" ], + notification_command = "sms-notification", + users = [ "icinga-oncall-2nd-level" ], + + times = { + begin = 30m, + end = 1h + } + }, + notifications["escalation-sms-1st-level"] = { + templates = [ "generic-notification" ], + notification_command = "sms-notification", + users = [ "icinga-oncall-1st-level" ], + + times = { + begin = 1h, + end = 2h + } + } + } + } + +> **Note** +> +> Instead of assigning users to notifications, you can also add the `user_groups` +> attribute with a list of user groups to the `Notification` object. Icinga 2 will +> resolve all group members and send notifications and notification escalations to +> all of them. + +### First Notification Delay + +Sometimes the problem in question should not be notified when the first notification +happens, but a defined time duration afterwards. In Icinga 2 you can use the `times` +dictionary and set `begin = 15m` as key and value if you want to suppress notifications +in the first 15 minutes. Leave out the `end` key - if not set, Icinga 2 will not check against any +end time for this notification. + + object Host "localhost" { + services["ping4"] = { + notifications["mail"] = { + templates = [ "generic-notification" ], + notification_command = "mail-notification", + users = [ "icingaadmin" ], + + times = { + begin = 15m // delay first notification + } + } + } + } + +> **Note** +> +> In Icinga 1.x the attribute is called `first_notification_delay`. + +### Notification Filters by State and Type + +If there are no notification state and type filter attributes defined at the `Notification` +or `User` object Icinga 2 assumes that all states and types are being notified. + +> **Note** +> +> In order to notify on problem states, you still need the type filter `NotificationFilterProblem`. + +Available state and type filters for notifications are: + + template Notification "generic-notification" { + + notification_state_filter = (StateFilterWarning | + StateFilterCritical | + StateFilterUnknown), + notification_type_filter = (NotificationFilterProblem | + NotificationFilterAcknowledgement | + NotificationFilterRecovery | + NotificationFilterCustom | + NotificationFilterFlappingStart | + NotificationFilterFlappingEnd | + NotificationFilterDowntimeStart | + NotificationFilterDowntimeEnd | + NotificationFilterDowntimeRemoved), + } +> **Note** +> +> If you are familiar with Icinga 1.x `notification_options` please note that they have been split +> into type and state, and allow more fine granular filtering for example on downtimes and flapping. +> You can filter for acknowledgements and custom notifications too. \ No newline at end of file diff --git a/doc/8-differences-between-icinga-1x-and-2.md b/doc/8-differences-between-icinga-1x-and-2.md index a2a504c16..94e695e8c 100644 --- a/doc/8-differences-between-icinga-1x-and-2.md +++ b/doc/8-differences-between-icinga-1x-and-2.md @@ -1,4 +1,4 @@ -# Differences between Icinga 1.x and 2 +# Differences between Icinga 1.x and 2 ## Configuration Format diff --git a/etc/icinga2/conf.d/notifications.conf b/etc/icinga2/conf.d/notifications.conf index c0f80da1d..9f58e558f 100644 --- a/etc/icinga2/conf.d/notifications.conf +++ b/etc/icinga2/conf.d/notifications.conf @@ -39,6 +39,7 @@ object NotificationCommand "mail-service-notification" inherits "plugin-notifica Additional Info: $SERVICEOUTPUT$ Comment: [$NOTIFICATIONAUTHORNAME$] $NOTIFICATIONCOMMENT$\"}}}, + "/bin/mail", "-s", "\"$NOTIFICATIONTYPE$ - $HOSTDISPLAYNAME$ - $SERVICEDISPLAYNAME$ is $SERVICESTATE$\"", "$USEREMAIL$" diff --git a/lib/icinga/notification.cpp b/lib/icinga/notification.cpp index 327c69f41..3f0ef4f3b 100644 --- a/lib/icinga/notification.cpp +++ b/lib/icinga/notification.cpp @@ -130,7 +130,7 @@ Dictionary::Ptr Notification::GetTimes(void) const unsigned long Notification::GetNotificationTypeFilter(void) const { if (m_NotificationTypeFilter.IsEmpty()) - return ~(unsigned long)0; /* All states. */ + return ~(unsigned long)0; /* All types. */ else return m_NotificationTypeFilter; } diff --git a/lib/icinga/user.cpp b/lib/icinga/user.cpp index 22f19bfd5..07157005f 100644 --- a/lib/icinga/user.cpp +++ b/lib/icinga/user.cpp @@ -101,7 +101,7 @@ TimePeriod::Ptr User::GetNotificationPeriod(void) const unsigned long User::GetNotificationTypeFilter(void) const { if (m_NotificationTypeFilter.IsEmpty()) - return ~(unsigned long)0; /* All states. */ + return ~(unsigned long)0; /* All types. */ else return m_NotificationTypeFilter; }