## Notifications
-TODO
-
Notifications on alerts are an integral part of your Icinga 2 monitoring application.
+There are many ways of getting a notification to the actual receiver - Email, XMPP,
+IRC, Twitter, etc. The default method for executing a notification command are
+plugin scripts used for notifications.
+These may either be shell commands to invoke a system call to the `mail` binary
+or your own script fetching available macro values and doing proper formatting
+before sending the notification.
+Other mechanism will require writing the notification string into an api processing
+it there (for example ticket system integration).
+
+Such notification plugins are available from community users and professionals for
+example on the [MonitoringExchange](http://www.monitoringexchange.org) or the
+[Icinga Wiki](https://wiki.icinga.org). Or you'll write your own and share it.
+
+A notification requires one or more users (and/or user groups) who will be notified
+in case. These users must have all macro attributes defined which will be used in
+the `NotificationCommand` on execution, for example `email` as macro dictionary key
+is referenced as `$USEREMAIL$`.
+
+The user `icingaadmin` in this examples will get notified only on `WARNING` and
+`CRITICAL` states and `problem` and `recovery` notification types.
+
+ object User "icingaadmin" {
+ display_name = "Icinga 2 Admin",
+ enable_notifications = 1,
+ notification_state_filter = (StateFilterWarning |
+ StateFilterCritical),
+ notification_type_filter = (NotificationFilterProblem |
+ NotificationFilterRecovery),
+ macros = {
+ "email" = "icinga@localhost",
+ "pager" = "+49123456789"
+ }
+ }
+
+> **Note**
+>
+> If you don't set the `notification_state_filter` and `notification_type_filter`
+> configuration attributes for the `User` object, all states and types will be
+> notified.
+
You should choose which information you (and your notified users) are interested in
case of emergency, and also which information does not provide any value to you and
your environment.
+> **Note**
+>
+> The chain of attribute inheritance including the (additive) macro dictionary for
+> notifications will allow granular macros for every specific use case, such as
+> `$mail$` or `$mobile$` as `User` macros available in `NotificationCommand`.
+
+ Service -> Notification -> Command -> User
+
+There are various macros available at runtime execution of the `NotificationCommand`.
+The example below may or may not fit your needs.
+
+ object NotificationCommand "mail-notification" inherits "plugin-notification-command" {
+ command = [
+ "/usr/bin/printf",
+ "\"%b\"",
+ {{{\"***** Icinga *****
+
+ Notification Type: $NOTIFICATIONTYPE$
+
+ Service: $SERVICEDESC$
+ Host: $HOSTALIAS$
+ Address: $HOSTADDRESS$
+ State: $SERVICESTATE$
+
+ Date/Time: $LONGDATETIME$
+
+ Additional Info: $SERVICEOUTPUT$
+
+ Comment: [$NOTIFICATIONAUTHORNAME$] $NOTIFICATIONCOMMENT$\"}}},
+ "/bin/mail",
+ "-s",
+ "\"$NOTIFICATIONTYPE$ - $HOSTDISPLAYNAME$ - $SERVICEDISPLAYNAME$ is $SERVICESTATE$\"",
+ "$USEREMAIL$"
+ ]
+ }
+
+> **Note**
+>
+> Alternatively you can use the `export_macros` attributes to export all required
+> macros into the environment.
+
+You can add all shared attributes to a `Notification` template which is inherited
+to the defined notifications. That way you'll save duplicated attributes in each
+`Notification` object. Attributes can be overridden locally.
+
+ template Notification "generic-notification" {
+ notification_interval = 15m,
+
+ notification_command = "mail-service-notification",
+
+ notification_state_filter = (StateFilterWarning |
+ StateFilterCritical |
+ StateFilterUnknown),
+ notification_type_filter = (NotificationFilterProblem |
+ NotificationFilterAcknowledgement |
+ NotificationFilterRecovery |
+ NotificationFilterCustom |
+ NotificationFilterFlappingStart |
+ NotificationFilterFlappingEnd |
+ NotificationFilterDowntimeStart |
+ NotificationFilterDowntimeEnd |
+ NotificationFilterDowntimeRemoved),
+
+ notification_period = "24x7"
+ }
+
+> **Note**
+>
+> The `TimePeriod` `24x7` is shipped as example configuration with Icinga 2.
+
+Use the `generic-notification` template for the `mail` notification defined
+inline to the host's service `ping4` and assign the `icingaadmin` user.
+
+ object Host "localhost" {
+ services["ping4"] = {
+ notifications["mail"] = {
+ templates = [ "generic-notification" ],
+ notification_command = "mail-notification",
+ users = [ "icingaadmin" ],
+ }
+ }
+ }
+
+Notifications can be defined in `Service` templates inherited to the objects.
+
+> **Note**
+>
+> Instead of assigning users to notifications, you can also add the `user_groups`
+> attribute with a list of user groups to the `Notification` object. Icinga 2 will
+> resolve all group members and send notifications to all of them.
+
### Notification Escalations
-TODO
+When a problem notification is sent and a problem still exists after re-notification
+you may want to escalate the problem to the next support level. A different approach
+is to configure the default notification by email, and escalate the problem via sms
+if not already solved.
+
+You can define notification start and end times as additional configuration
+attributes making the `Notification` object a so-called `notification escalation`.
+Using templates you can share the basic notification attributes such as users or the
+`notification_interval` (and override them for the escalation then).
+
+Using the example from above, you can define additional users being escalated for sms
+notifications between start and end time.
+
+> **Note**
+>
+> `notification_state_filter` and `notification_type_filter` configuration attributes
+> are not set in this example.
+
+ object User "icinga-oncall-2nd-level" {
+ display_name = "Icinga 2nd Level",
+ enable_notifications = 1,
+
+ macros = {
+ "mobile" = "+49123456781"
+ }
+ }
+
+ object User "icinga-oncall-1st-level" {
+ display_name = "Icinga 1st Level",
+ enable_notifications = 1,
+
+ macros = {
+ "mobile" = "+49123456782"
+ }
+ }
+
+Define an additional `NotificationCommand` for sms notifications.
+
+> **Note**
+>
+> The example is not complete as there are many different sms providers.
+> Please note that sending sms notifications will require an sms provider
+> or local hardware with a sim card active.
+
+ object NotificationCommand "sms-notification" {
+ command = "$plugindir$/send_sms_notification $mobile$ ..."
+ }
+
+The two new notification escalations are added onto the host `localhost`
+and its service `ping4` using the `generic-notification` template.
+The user `icinga-oncall-2nd-level` will get notified by sms (`sms-notification`
+command) after `30m` until `1h`.
+
+> **Note**
+>
+> The `notification_interval` was set to 15m in the `generic-notification`
+> template example. Lower that value in your escalations by using a secondary
+> template or overriding the attribute directly in the `notifications` array
+> position for `escalation-sms-2nd-level`.
+
+If the problem does not get resolved or acknowledged preventing further notifications
+the `escalation-sms-1st-level` user will be escalated `1h` after the initial problem was
+notified, but only for one hour (`2h` as `end` key for the `times` dictionary).
+
+ object Host "localhost" {
+ services["ping4"] = {
+ notifications["mail"] = {
+ templates = [ "generic-notification" ],
+ notification_command = "mail-notification",
+ users = [ "icingaadmin" ],
+ },
+ notifications["escalation-sms-2nd-level"] = {
+ templates = [ "generic-notification" ],
+ notification_command = "sms-notification",
+ users = [ "icinga-oncall-2nd-level" ],
+
+ times = {
+ begin = 30m,
+ end = 1h
+ }
+ },
+ notifications["escalation-sms-1st-level"] = {
+ templates = [ "generic-notification" ],
+ notification_command = "sms-notification",
+ users = [ "icinga-oncall-1st-level" ],
+
+ times = {
+ begin = 1h,
+ end = 2h
+ }
+ }
+ }
+ }
+
+> **Note**
+>
+> Instead of assigning users to notifications, you can also add the `user_groups`
+> attribute with a list of user groups to the `Notification` object. Icinga 2 will
+> resolve all group members and send notifications and notification escalations to
+> all of them.
+
+### First Notification Delay
+
+Sometimes the problem in question should not be notified when the first notification
+happens, but a defined time duration afterwards. In Icinga 2 you can use the `times`
+dictionary and set `begin = 15m` as key and value if you want to suppress notifications
+in the first 15 minutes. Leave out the `end` key - if not set, Icinga 2 will not check against any
+end time for this notification.
+
+ object Host "localhost" {
+ services["ping4"] = {
+ notifications["mail"] = {
+ templates = [ "generic-notification" ],
+ notification_command = "mail-notification",
+ users = [ "icingaadmin" ],
+
+ times = {
+ begin = 15m // delay first notification
+ }
+ }
+ }
+ }
+
+> **Note**
+>
+> In Icinga 1.x the attribute is called `first_notification_delay`.
+
+### Notification Filters by State and Type
+
+If there are no notification state and type filter attributes defined at the `Notification`
+or `User` object Icinga 2 assumes that all states and types are being notified.
+
+> **Note**
+>
+> In order to notify on problem states, you still need the type filter `NotificationFilterProblem`.
+
+Available state and type filters for notifications are:
+
+ template Notification "generic-notification" {
+
+ notification_state_filter = (StateFilterWarning |
+ StateFilterCritical |
+ StateFilterUnknown),
+ notification_type_filter = (NotificationFilterProblem |
+ NotificationFilterAcknowledgement |
+ NotificationFilterRecovery |
+ NotificationFilterCustom |
+ NotificationFilterFlappingStart |
+ NotificationFilterFlappingEnd |
+ NotificationFilterDowntimeStart |
+ NotificationFilterDowntimeEnd |
+ NotificationFilterDowntimeRemoved),
+ }
+> **Note**
+>
+> If you are familiar with Icinga 1.x `notification_options` please note that they have been split
+> into type and state, and allow more fine granular filtering for example on downtimes and flapping.
+> You can filter for acknowledgements and custom notifications too.
\ No newline at end of file