## What is Icinga 2?
-Icinga 2 is an enterprise-grade open source monitoring system which keeps watch over networks and any conceivable network resource, notifies the user of errors and recoveries and generates performance data for reporting. Scalable and extensible, Icinga can monitor complex, large environments across dispersed locations.
+Icinga 2 is an enterprise-grade open source monitoring system which keeps watch over networks
+and any conceivable network resource, notifies the user of errors and recoveries and generates
+performance data for reporting. Scalable and extensible, Icinga 2 can monitor complex, large
+environments across dispersed locations.
## Licensing
## Support
Support for Icinga 2 is available in a number of ways. Please have a look at
-the support overview page at https://www.icinga.org/support/.
+the support overview page at [https://support.icinga.org].
## What's New in Version 0.0.3
-* Lots of things.
+* `StatusDataWriter` and `ExternalCommandListener` (former `Compat`) and `CompatLogger`
+(former CompatLog) for status.dat/objects.cache/icinga2.cmd/icinga.log for Icinga 1.x Classic UI support
+* `IdoMysqlConnection` and `ExternalCommandListener` for Icinga 1.x Web
+* `IdoMysqlConnection` for Icinga 1.x Reporting, NagVis
+* `LivestatusListener` for addons using the livestatus interface (history tables tbd)
+* `PerfDataWriter` for graphing addons such as PNP/inGraph/graphite (can be loaded multiple times!)
+* `GraphiteWriter` for sending metrics to directly to graphite carbon sockets
+* `CheckResultReader` to collect Icinga 1.x slave checkresults (migrate your distributed setup step-by-step)
+* `ClusterListener` for real distributed architecture including config and runtime data (checks, comments, downtimes) sync and replay
+* `SyslogLogger`, `FileLogger` and `ConsoleLogger` for different types of logging
+* Domain support for distributed cluster environments
+* Config migration script supporting easier migration from Icinga 1.x configuration
+* Reviewed configuration options, additional attributes added
+* Enhanced ITL, added sample configuration
+* Enable/Disable Icinga 2 features on CLI
+* Documentation using Markdown (`Getting Started`, `Monitoring Basics`, `Object Types`, `Icinga Template Library`,
+`Advanced Topics`, `Migration from Icinga 1.x`, `Differences between Icinga 1.x and 2`, `Vagrant Demo VM`)
+* Vagrant Demo VM supported by Puppet modules installing RPM snapshots for Icinga 2, Icinga 1.x Classic UI and Web
+* Package snapshots available on [packages.icinga.org]
+
*/
object Host "localhost" {
services["ping4"] = {
+ templates = [ "generic-service" ],
+
check_command = "ping4"
},
services["ping6"] = {
+ templates = [ "generic-service" ],
+
check_command = "ping6"
},
services["http"] = {
+ templates = [ "generic-service" ],
+
check_command = "http_ip"
},
services["ssh"] = {
+ templates = [ "generic-service" ],
+
check_command = "ssh"
},
services["load"] = {
+ templates = [ "generic-service" ],
+
check_command = "load"
},
services["processes"] = {
+ templates = [ "generic-service" ],
+
check_command = "processes"
},
services["users"] = {
+ templates = [ "generic-service" ],
+
check_command = "users"
},
services["disk"] = {
+ templates = [ "generic-service" ],
+
check_command = "disk"
},
You can download and build Icinga 1.x Classic UI in standalone mode using the
following commands:
- $ wget http://downloads.sourceforge.net/project/icinga/icinga/1.9.3/icinga-1.9.3.tar.gz
- $ tar xzf icinga-1.9.3.tar.gz ; cd icinga-1.9.3
+ $ wget http://downloads.sourceforge.net/project/icinga/icinga/1.10.0/icinga-1.10.0.tar.gz
+ $ tar xzf icinga-1.10.0.tar.gz ; cd icinga-1.10.0
$ ./configure --enable-classicui-standalone --prefix=/usr/local/icinga2-classicui
$ make classicui-standalone
Once you've built Classic UI you can install it:
- # make install-classicui-standalone install-webconf-auth
+ # make install-classicui-standalone install-classicui-standalone-conf install-webconf-auth
# service apache2 restart
> **Note**
update the `plugindir` macro in your Icinga 2 configuration. This macro is used
by the service templates contained in the Icinga Template Library to determine
where to find the plugin binaries.
+
+### Integrate Additonal Plugins
+
+You may require a custom check plugin not provided by the official Nagios plugins.
+All existing Nagios or Icinga 1.x plugins found on public community websites
+will work with Icinga 2 as well.
+
+* [MonitoringExchange](https://www.monitoringexchange.org)
+* [Icinga Wiki]()
+
+Once you have downloaded the plugin copy them into the directory defined by the global
+`IcingaMacro` `$plugindir$` and make sure that the user the Icinga daemon is running as
+can execute the the plugin binary. Plugins should provide the `--help` parameter
+providing details how they must get called in your command definition later on.
+
+ # cp check_snmp_int.pl /usr/local/icinga/libexec/
+ # chmod +x /usr/local/icinga/libexec/check_snmp_int.pl
+
+ # /usr/local/icinga/libexec/check_snmp_int.pl --help
+
+> **Note**
+>
+> You may require additional libraries for scripts. Please consult the installation
+> documentation and/or README for their installation requirements.
> **Note**
>
> The Icinga 2 RPM packages install the schema files into
-> `/usr/share/doc/icinga2-ido-mysql-*/schema`. The Icinga 2 dist tarball ships
-> the schema files in `components/db_ido_mysql/schema/`.
+> `/usr/share/doc/icinga2-ido-mysql-*/schema` (`*` means package version).
+> The Icinga 2 dist tarball ships the schema files in `components/db_ido_mysql/schema/`.
### Upgrading the database
## Groups
-TODO
+Groups are used for combining hosts, services and users into
+accessible configuration attributes and views in external (web)
+interfaces.
+
+Group membership is defined at the respective object itself. If
+you have a hostgroup name `windows` for example, and want to assign
+specific hosts to this group for later viewing the group on your
+alert dashboard, first create the hostgroup:
+
+ object HostGroup "windows" {
+ display_name = "Windows Servers"
+ }
+
+Then add your hosts to this hostgroup
+
+ object Host "mssql-srv1" {
+ groups = [ "windows" ],
+ macros = {
+ "mssql_port" = 1433
+ }
+ }
+
+ object Host "mssql-srv2" {
+ groups = [ "windows" ],
+ macros = {
+ "mssql_port" = 1433
+ }
+ }
+
+> **Best Practice**
+>
+> Add the hostgroup membership into generic templates combined with
+> other attributes and let all windows hosts inherit from that template.
+> You can also define multiple group memberships.
+
+ template Host "windows-mssql-template" {
+ groups = [ "windows" ],
+ macros = {
+ "mssql_port" = 1433
+ }
+ }
+
+ object Host "mssql-srv1" {
+ templates = [ "windows-mssql-template" ]
+ }
+
+ object Host "mssql-srv2" {
+ templates = [ "windows-mssql-template" ]
+ }
+
+This can be done for service and user groups the same way. Additionally
+the user groups are associated as attributes in `Notification` objects.
+
+ object UserGroup "windows-mssql-admins" {
+ display_name = "Windows MSSQL Admins"
+ }
+
+ template User "generic-windows-mssql-users" {
+ groups = [ "windows-mssql-admins" ]
+ }
+
+ object User "win-mssql-noc" inherits "generic-windows-mssql-users" {
+ macros = {
+ "email" = "noc@company.com"
+ }
+ }
+
+ object User "win-mssql-ops" inherits "generic-windows-mssql-users" {
+ macros = {
+ "email" = "ops@company.com"
+ }
+ }
+
+ object Host "localhost" {
+ services["ping4"] = {
+ notifications["mail"] = {
+ templates = [ "generic-notification" ],
+ notification_command = "mail-notification",
+ user_groups = [ "windows-admins" ],
+ }
+ }
+ }
## Time Periods
-TODO
+Time Periods define time ranges in Icinga where event actions are
+triggered, for example if a service check is executed or not within
+the `check_period` attribute. Or a notification should be sent to
+users or not, filtered by the `notification_period` configuration
+attribute for `Notification` and `User` objects.
+
+> **Note**
+>
+> If you are familar with Icinga 1.x - these time period definitions
+> are called `legacy timeperiods` in Icinga 2.
+>
+> An Icinga 2 legacy timeperiod requires the `ITL` provided template
+>`legacy-timeperiod`.
+
+The `TimePeriod` attribute `ranges` may contain multiple directives,
+including weekdays, days of the month, and calendar dates.
+These types may overlap/override other types in your ranges dictionary.
+
+The descending order of precedence is as follows:
+
+* Calendar date (2008-01-01)
+* Specific month date (January 1st)
+* Generic month date (Day 15)
+* Offset weekday of specific month (2nd Tuesday in December)
+* Offset weekday (3rd Monday)
+* Normal weekday (Tuesday)
+
+If you don't set any `check_period` or `notification_period` attribute
+on your configuration objects Icinga 2 assumes `24x7` as time period
+as shown below.
+
+ object TimePeriod "24x7" inherits "legacy-timeperiod" {
+ display_name = "Icinga 2 24x7 TimePeriod",
+ ranges = {
+ "monday" = "00:00-24:00",
+ "tuesday" = "00:00-24:00",
+ "wednesday" = "00:00-24:00",
+ "thursday" = "00:00-24:00",
+ "friday" = "00:00-24:00",
+ "saturday" = "00:00-24:00",
+ "sunday" = "00:00-24:00",
+ }
+ }
+
+If your operation staff should only be notified during workhours
+create a new timeperiod named `workhours` defining a work day with
+09:00 to 17:00.
+
+ object TimePeriod "workhours" inherits "legacy-timeperiod" {
+ display_name = "Icinga 2 8x5 TimePeriod",
+ ranges = {
+ "monday" = "09:00-17:00",
+ "tuesday" = "09:00-17:00",
+ "wednesday" = "09:00-17:00",
+ "thursday" = "09:00-17:00",
+ "friday" = "09:00-17:00",
+ }
+ }
+
+Assign the timeperiod as `notification_period` to the `Notification`
+object then.
+
+ object Host "localhost" {
+ services["ping4"] = {
+ notifications["mail"] = {
+ templates = [ "generic-notification" ],
+ notification_command = "mail-notification",
+ users = [ "icingaadmin" ],
+ notification_period = "workhours"
+ }
+ }
+ }
## Event Handlers
-TODO
+Event handlers are defined as `EventCommand` objects in Icinga 2.
+
+Unlike notifications event commands are called on every service state change
+if defined. Therefore the `EventCommand` object should define a command line
+evaluating the current service state and other service runtime attributes
+available through runtime macros. Runtime macros such as `$SERVICESTATETYPE$`
+and `$SERVICESTATE$` will be processed by Icinga 2 helping on fine-granular
+events being triggered.
+
+Common use case scenarios are a failing HTTP check requiring an immediate
+restart via event command, or a an application is locked and requires
+a restart on detection.
+
+
## Logging
-TODO
+Icinga 2 supports three different types of logging:
+
+* File logging (local Icinga 2 log file)
+* Syslog facility logging (system's syslog application)
+* Console logging (`STDOUT` on tty)
+
+> **Note**
+>
+> You can define multiple logging objects at once.
+
+The most common scenario will be logging Icinga 2's output to
+syslog with severity `information`.
+
+ object SyslogLogger "syslog" {
+ severity = "information"
+ }
+
+For debugging purposes you can install a `FileLogger` object
+and forward the `debug` serverity into an Icinga 2 debug file.
+
+ object FileLogger "debug-file" {
+ severity = "debug",
+ path = "/var/log/icinga2/icinga2-debug.log"
+ }
+
+If you're starting Icinga 2 not as daemon, but in foreground
+using `/usr/bin/icinga2 -c /etc/icinga2/icinga2.conf` you may
+want to enable console logging temporary too.
+
+ object ConsoleLogger "debug-console" {
+ severity = "debug"
+ }
+
+
+
## Performance Data
-TODO
+When a service check is executed plugins should provide so-called
+`performance data`. Next to that additional check performance data
+can be fetched using Icinga 2 runtime macros such as the check latency
+or the current service state.
+
+The performance data may be passed to external applications which
+then generate nice graphs for historical reporting and trending.
+
+Well-known addons processing Icinga performance data are PNP4Nagios,
+inGraph and Graphite.
+
+> **Note**
+>
+> As there are no real host checks in Icinga 2, there is no performance
+> data generated for hosts.
+
+### Writing Performance Data Files
+
+PNP4Nagios and inGraph use performance data collector daemons to fetch
+the current performance files for their backend updates.
+
+Therefore the Icinga 2 `PerfdataWriter` object allows you to define
+the output template format backed with Icinga 2 runtime macros.
+
+ format_template = "DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICESTATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$",
+
+The default template is already provided with the Icinga 2 feature configuration
+which can be enabled using
+
+ # i2enfeature perfdata
+
+By default all performance data files are rotated in a 15 seconds interval into
+the `/var/spool/icinga2/perfdata/` directory as `service-perfdata.<timestamp>`.
+External collectors need to parse the rotated performance data files and then
+remove the processed files.
+
+> **Note**
+>
+> If you enable this feature by accident and no collector daemon is processing
+> these files, the filesystem's inodes may run out. Make sure to watch the
+> `localhost` service check `disks` and define your own inode warning
+> and critical thresholds.
+
+
+### Graphite Carbon Cache Writer
+
+While there are some Graphite collector scripts for Icinga 1.x available it's
+more reasonable to directly process the check and plugin performance in memory
+in Icinga 2. Once there are new metrics available, Icinga 2 will directly
+write these to the defined Graphite Carbon daemon tcp socket.
+
+You can enable the feature using
+
+ # i2enfeature graphite
+
+The `GraphiteWriter` object expects the Graphite Carbon Cache socket listening
+at `127.0.0.1` on port `2003` by default.
+
+The current naming schema is
+
+ icinga.<hostname>.<servicename>.<metricname>
## Status Data
-TODO
+Icinga 1.x writes object configuration data and status data in a cyclic
+interval to its `objects.cache` and `status.dat` files. Icinga 2 provides
+the `StatusDataWriter` object which dumps all configuration objects and
+status updates in a regular interval.
+
+ # i2enfeature statusdata
+
+Icinga 1.x Classic UI requires this data set as part of its backend.
+
+> **Note**
+>
+> If you are not using any web interface or addon requiring this output
+> do not enable this feature.
## Compat Logging
+
+The Icinga 1.x log format is considered being the `Compat Log`
+in Icinga 2 provided with the `CompatLogger` object.
+
+These logs are not only used for informal representation in
+external web interfaces parsing the logs, but also to generate
+sla reports and trends in Icinga 1.x Classic UI.
+
+The `CompatLogger` object can be enabled with
+
+ # i2enfeature compatlog
+
+By default, the Icinga 1.x log file called `icinga.log` is located
+in `/var/log/icinga2/compat`. Rotated log files are moved into
+`var/log/icinga2/compat/archives`.
+
+The format cannot be changed without breaking compatibility to
+existing log parsers.
+
+ # tail -f /var/log/icinga2/compat/icinga.log
+
+ [1382115688] LOG ROTATION: HOURLY
+ [1382115688] LOG VERSION: 2.0
+ [1382115688] HOST STATE: CURRENT;localhost;UP;HARD;1;
+ [1382115688] SERVICE STATE: CURRENT;localhost;disk;WARNING;HARD;1;
+ [1382115688] SERVICE STATE: CURRENT;localhost;http;OK;HARD;1;
+ [1382115688] SERVICE STATE: CURRENT;localhost;load;OK;HARD;1;
+ [1382115688] SERVICE STATE: CURRENT;localhost;ping4;OK;HARD;1;
+ [1382115688] SERVICE STATE: CURRENT;localhost;ping6;OK;HARD;1;
+ [1382115688] SERVICE STATE: CURRENT;localhost;processes;WARNING;HARD;1;
+ [1382115688] SERVICE STATE: CURRENT;localhost;ssh;OK;HARD;1;
+ [1382115688] SERVICE STATE: CURRENT;localhost;users;OK;HARD;1;
+ [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;disk;1382115705
+ [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;http;1382115705
+ [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;load;1382115705
+ [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;ping4;1382115705
+ [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;ping6;1382115705
+ [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;processes;1382115705
+ [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;ssh;1382115705
+ [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;users;1382115705
+ [1382115731] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;localhost;ping6;2;critical test|
+ [1382115731] SERVICE ALERT: localhost;ping6;CRITICAL;SOFT;2;critical test
+
## Check Result Files
-TODO
+Icinga 1.x writes its check result files into a temporary spool directory
+where it reads these check result files in a regular interval from.
+While this is extremly inefficient in performance regards it has been
+rendered useful for passing passive check results directly into Icinga 1.x
+skipping the external command pipe.
+
+Several clustered/distributed environments and check-aggregation addons
+use that method. In order to support step-by-step migration of these
+environments, Icinga 2 ships the `CheckResultReader` object.
+
+There is no feature configuration available, but it must be defined
+on-demand in your Icinga 2 objects configuration.
+
+ object CheckResultReader "reader" {
+ spool_dir = "/data/check-results"
+ }
+
+
+
enable\_passive\_checks|**Optional.** Whether passive checks are enabled. Defaults to true.
enable\_event\_handler|**Optional.** Enables event handlers for this service. Defaults to true.
event\_command |**Optional.** The name of an event command that should be executed every time the service's state changes.
- flapping\_threshold|TODO
- volatile |TODO
+ flapping\_threshold|**Optional.** The flapping threshold in percent when a service is considered to be flapping.
+ volatile |**Optional.** The volatile setting enables always `HARD` state types if `NOT-OK` state changes occur.
host_dependencies|**Optional.** A list of host names which this host depends on. These dependencies are used to determine whether the host is unreachable.
service_dependencies|**Optional.** A list of services which this host depends on. Each array element must be a dictionary containing the keys "host" and "service". These dependencies are used to determine whether the host is unreachable.
groups |**Optional.** The service groups this service belongs to.
macros |**Optional.** A dictionary containing macros that are specific to this notification object.
users |**Optional.** A list of user names who should be notified.
user_groups |**Optional.** A list of user group names who should be notified.
- times |TODO
- notification_command|TODO
- notification_interval|TODO
- notification_period|TODO
- notification_type_filter|TODO
- notification_state_filter|TODO
+ times |**Optional.** A dictionary containing `begin` and `end` attributes for the notification.
+ notification_command|**Required.** The name of the notification command which should be executed when the notification is triggered.
+ notification_interval|**Optional.** The notification interval (in seconds). This interval is used for active notifications. Defaults to 5 minutes.
+ notification_period|**Optional.** The name of a time period which determines when this notification should be triggered. Not set by default.
+ notification_type_filter|**Optional.** A set of state filters when this notification should be triggered. By default everything is matched.
+ notification_state_filter|**Optional.** A set of type filters when this notification should be triggered. By default everything is matched.
+
+Available notification type and state filters:
+
+ StateFilterOK
+ StateFilterWarning
+ StateFilterCritical
+ StateFilterUnknown
+
+ NotificationFilterDowntimeStart
+ NotificationFilterDowntimeEnd
+ NotificationFilterDowntimeRemoved
+ NotificationFilterCustom
+ NotificationFilterAcknowledgement
+ NotificationFilterProblem
+ NotificationFilterRecovery
+ NotificationFilterFlappingStart
+ NotificationFilterFlappingEnd
+
+> **Note**
+>
+> In order to notify on problem states, you will need the type filter `NotificationFilterProblem`.
### <a id="objecttype-user"></a> User
}
}
+Available notification type and state filters:
+
+ StateFilterOK
+ StateFilterWarning
+ StateFilterCritical
+ StateFilterUnknown
+
+ NotificationFilterDowntimeStart
+ NotificationFilterDowntimeEnd
+ NotificationFilterDowntimeRemoved
+ NotificationFilterCustom
+ NotificationFilterAcknowledgement
+ NotificationFilterProblem
+ NotificationFilterRecovery
+ NotificationFilterFlappingStart
+ NotificationFilterFlappingEnd
+
+> **Note**
+>
+> In order to notify on problem states, you will need the type filter `NotificationFilterProblem`.
Attributes:
custom |**Optional.** A dictionary containing custom attributes that are specific to this user.
groups |**Optional.** An array of group names.
enable_notifications|**Optional.** Whether notifications are enabled for this user.
- notification_period|TODO
- notification_type_filter|TODO
- notification_state_filter|TODO
+ notification_period|**Optional.** The name of a time period which determines when this notification should be triggered. Not set by default.
+ notification_type_filter|**Optional.** A set of state filters when this notification should be triggered. By default everything is matched.
+ notification_state_filter|**Optional.** A set of type filters when this notification should be triggered. By default everything is matched.
### <a id="objecttype-usergroup"></a> UserGroup
----------------|----------------
display_name |**Optional.** A short description of the time period.
methods |**Required.** The "update" script method takes care of updating the internal representation of the time period. In virtually all cases you should just inherit from the "legacy-timeperiod" template to take care of this setting.
- ranges |TODO
+ ranges |**Required.** A dictionary containing information which days and durations apply to this timeperiod.
The `/etc/icinga2/conf.d/timeperiods.conf` file is usually used to define
timeperiods including this one.
### <a id="objecttype-clusterlistener"></a> ClusterListener
-TODO
+ClusterListener objects are used to specify remote cluster
+node peers and the certificate files used for ssl
+authorization.
Example:
----------------|----------------
node |**Required.** The hostname/IP address of the remote Icinga 2 instance.
service |**Required.** The service name/port of the remote Icinga 2 instance.
- config\_files |TODO
+ config\_files |**Optional.** A list of configuration files sent to remote peers (wildcards possible).
accept\_config |**Optional.** A list of endpoint names from which this endpoint accepts configuration files.
### <a id="objecttype-domain"></a> Domain
## Volatile Services
-By default all services remain in a non-volatile state. Whe a problem
+By default all services remain in a non-volatile state. When a problem
occurs, the `SOFT` state applies and once `max_check_attempts` attribute
is reached with the check counter, a `HARD` state transition happens.
Notifications are only triggered by `HARD` state changes and are then
projects / icinga2 / commitdiff