1 # <a id="advanced-topics"></a> Advanced Topics
3 This chapter covers a number of advanced topics. If you're new to Icinga you
4 can safely skip over things you're not interested in.
6 ## <a id="downtimes"></a> Downtimes
8 Downtimes can be scheduled for planned server maintenance or
9 any other targetted service outage you are aware of in advance.
11 Downtimes will suppress any notifications, and may trigger other
12 downtimes too. If the downtime was set by accident, or the duration
13 exceeds the maintenance, you can manually cancel the downtime.
14 Planned downtimes will also be taken into account for SLA reporting
15 tools calculating the SLAs based on the state and downtime history.
17 Multiple downtimes for a single object may overlap. This is useful
18 when you want to extend your maintenance window taking longer than expected.
19 If there are multiple downtimes triggered for one object, the overall downtime depth
20 will be greater than `1`.
23 If the downtime was scheduled after the problem changed to a critical hard
24 state triggering a problem notification, and the service recovers during
25 the downtime window, the recovery notification won't be suppressed.
27 ### <a id="fixed-flexible-downtimes"></a> Fixed and Flexible Downtimes
29 A `fixed` downtime will be activated at the defined start time, and
30 removed at the end time. During this time window the service state
31 will change to `NOT-OK` and then actually trigger the downtime.
32 Notifications are suppressed and the downtime depth is incremented.
34 Common scenarios are a planned distribution upgrade on your linux
35 servers, or database updates in your warehouse. The customer knows
36 about a fixed downtime window between 23:00 and 24:00. After 24:00
37 all problems should be alerted again. Solution is simple -
38 schedule a `fixed` downtime starting at 23:00 and ending at 24:00.
40 Unlike a `fixed` downtime, a `flexible` downtime will be triggered
41 by the state change in the time span defined by start and end time,
42 and then last for the specified duration in minutes.
44 Imagine the following scenario: Your service is frequently polled
45 by users trying to grab free deleted domains for immediate registration.
46 Between 07:30 and 08:00 the impact will hit for 15 minutes and generate
47 a network outage visible to the monitoring. The service is still alive,
48 but answering too slow to Icinga 2 service checks.
49 For that reason, you may want to schedule a downtime between 07:30 and
50 08:00 with a duration of 15 minutes. The downtime will then last from
51 its trigger time until the duration is over. After that, the downtime
52 is removed (may happen before or after the actual end time!).
54 ### <a id="scheduling-downtime"></a> Scheduling a downtime
56 This can either happen through a web interface or by sending an [external command](5-advanced-topics.md#external-commands)
57 to the external command pipe provided by the `ExternalCommandListener` configuration.
59 Fixed downtimes require a start and end time (a duration will be ignored).
60 Flexible downtimes need a start and end time for the time span, and a duration
61 independent from that time span.
63 ### <a id="triggered-downtimes"></a> Triggered Downtimes
65 This is optional when scheduling a downtime. If there is already a downtime
66 scheduled for a future maintenance, the current downtime can be triggered by
67 that downtime. This renders useful if you have scheduled a host downtime and
68 are now scheduling a child host's downtime getting triggered by the parent
69 downtime on NOT-OK state change.
71 ### <a id="recurring-downtimes"></a> Recurring Downtimes
73 [ScheduledDowntime objects](6-object-types.md#objecttype-scheduleddowntime) can be used to set up
74 recurring downtimes for services.
78 apply ScheduledDowntime "backup-downtime" to Service {
79 author = "icingaadmin"
80 comment = "Scheduled downtime for backup"
83 monday = "02:00-03:00"
84 tuesday = "02:00-03:00"
85 wednesday = "02:00-03:00"
86 thursday = "02:00-03:00"
87 friday = "02:00-03:00"
88 saturday = "02:00-03:00"
89 sunday = "02:00-03:00"
92 assign where "backup" in service.groups
96 ## <a id="comments-intro"></a> Comments
98 Comments can be added at runtime and are persistent over restarts. You can
99 add useful information for others on repeating incidents (for example
100 "last time syslog at 100% cpu on 17.10.2013 due to stale nfs mount") which
101 is primarly accessible using web interfaces.
103 Adding and deleting comment actions are possible through the external command pipe
104 provided with the `ExternalCommandListener` configuration. The caller must
105 pass the comment id in case of manipulating an existing comment.
108 ## <a id="acknowledgements"></a> Acknowledgements
110 If a problem is alerted and notified you may signal the other notification
111 recipients that you are aware of the problem and will handle it.
113 By sending an acknowledgement to Icinga 2 (using the external command pipe
114 provided with `ExternalCommandListener` configuration) all future notifications
115 are suppressed, a new comment is added with the provided description and
116 a notification with the type `NotificationFilterAcknowledgement` is sent
117 to all notified users.
119 ### <a id="expiring-acknowledgements"></a> Expiring Acknowledgements
121 Once a problem is acknowledged it may disappear from your `handled problems`
122 dashboard and no-one ever looks at it again since it will suppress
125 This `fire-and-forget` action is quite common. If you're sure that a
126 current problem should be resolved in the future at a defined time,
127 you can define an expiration time when acknowledging the problem.
129 Icinga 2 will clear the acknowledgement when expired and start to
130 re-notify if the problem persists.
133 ## <a id="timeperiods"></a> Time Periods
135 Time Periods define time ranges in Icinga where event actions are
136 triggered, for example whether a service check is executed or not within
137 the `check_period` attribute. Or a notification should be sent to
138 users or not, filtered by the `period` and `notification_period`
139 configuration attributes for `Notification` and `User` objects.
143 > If you are familar with Icinga 1.x - these time period definitions
144 > are called `legacy timeperiods` in Icinga 2.
146 > An Icinga 2 legacy timeperiod requires the `ITL` provided template
147 >`legacy-timeperiod`.
149 The `TimePeriod` attribute `ranges` may contain multiple directives,
150 including weekdays, days of the month, and calendar dates.
151 These types may overlap/override other types in your ranges dictionary.
153 The descending order of precedence is as follows:
155 * Calendar date (2008-01-01)
156 * Specific month date (January 1st)
157 * Generic month date (Day 15)
158 * Offset weekday of specific month (2nd Tuesday in December)
159 * Offset weekday (3rd Monday)
160 * Normal weekday (Tuesday)
162 If you don't set any `check_period` or `notification_period` attribute
163 on your configuration objects Icinga 2 assumes `24x7` as time period
166 object TimePeriod "24x7" {
167 import "legacy-timeperiod"
169 display_name = "Icinga 2 24x7 TimePeriod"
171 "monday" = "00:00-24:00"
172 "tuesday" = "00:00-24:00"
173 "wednesday" = "00:00-24:00"
174 "thursday" = "00:00-24:00"
175 "friday" = "00:00-24:00"
176 "saturday" = "00:00-24:00"
177 "sunday" = "00:00-24:00"
181 If your operation staff should only be notified during workhours
182 create a new timeperiod named `workhours` defining a work day from
185 object TimePeriod "workhours" {
186 import "legacy-timeperiod"
188 display_name = "Icinga 2 8x5 TimePeriod"
190 "monday" = "09:00-17:00"
191 "tuesday" = "09:00-17:00"
192 "wednesday" = "09:00-17:00"
193 "thursday" = "09:00-17:00"
194 "friday" = "09:00-17:00"
198 Use the `period` attribute to assign time periods to
199 `Notification` and `Dependency` objects:
201 object Notification "mail" {
202 import "generic-notification"
204 host_name = "localhost"
206 command = "mail-notification"
207 users = [ "icingaadmin" ]
211 ## <a id="use-functions-object-config"></a> Use Functions in Object Configuration
213 There is a limited scope where functions can be used as object attributes such as:
215 * As value for [Custom Attributes](3-monitoring-basics.md#custom-attributes-functions)
216 * Returning boolean expressions for [set_if](5-advanced-topics.md#use-functions-command-arguments-setif) inside command arguments
217 * Returning a [command](5-advanced-topics.md#use-functions-command-attribute) array inside command objects
219 The other way around you can create objects dynamically using your own global functions.
223 > Functions called inside command objects share the same global scope as runtime macros.
224 > Therefore you can access host custom attributes like `host.vars.os`, or any other
225 > object attribute from inside the function definition used for [set_if](5-advanced-topics.md#use-functions-command-arguments-setif) or [command](5-advanced-topics.md#use-functions-command-attribute).
227 Tips when implementing functions:
229 * Use [log()](20-library-reference.md#global-functions) to dump variables. You can see the output
230 inside the `icinga2.log` file depending in your log severity
231 * Use the `icinga2 console` to test basic functionality (e.g. iterating over a dictionary)
232 * Build them step-by-step. You can always refactor your code later on.
234 ### <a id="use-functions-command-arguments-setif"></a> Use Functions in Command Arguments set_if
236 The `set_if` attribute inside the command arguments definition in the
237 [CheckCommand object definition](6-object-types.md#objecttype-checkcommand) is primarly used to
238 evaluate whether the command parameter should be set or not.
240 By default you can evaluate runtime macros for their existance, and if the result is not an empty
241 string the command parameter is passed. This becomes fairly complicated when want to evaluate
242 multiple conditions and attributes.
244 The following example was found on the community support channels. The user had defined a host
245 dictionary named `compellent` with the key `disks`. This was then used inside service apply for rules.
247 object Host "dict-host" {
248 check_command = "check_compellent"
249 vars.compellent["disks"] = {
250 file = "/var/lib/check_compellent/san_disks.0.json",
255 The more significant problem was to only add the command parameter `--disk` to the plugin call
256 when the dictionary `compellent` contains the key `disks`, and omit it if not found.
258 By defining `set_if` as [abbreviated lambda function](19-language-reference.md#nullary-lambdas)
259 and evaluating the host custom attribute `compellent` containing the `disks` this problem was
262 object CheckCommand "check_compellent" {
263 import "plugin-check-command"
264 command = [ "/usr/bin/check_compellent" ]
268 var host_vars = host.vars
270 var compel = host_vars.compellent
272 compel.contains("disks")
278 This implementation uses the dictionary type method [contains](20-library-reference.md#dictionary-contains)
279 and will fail if `host.vars.compellent` is not of the type `Dictionary`.
280 Therefore you can extend the checks using the [typeof](19-language-reference.md#types) function.
282 You can test the types using the `icinga2 console`:
285 Icinga (version: v2.3.0-193-g3eb55ad)
286 <1> => srv_vars.compellent["check_a"] = { file="outfile_a.json", checks = [ "disks", "fans" ] }
288 <2> => srv_vars.compellent["check_b"] = { file="outfile_b.json", checks = [ "power", "voltages" ] }
290 <3> => typeof(srv_vars.compellent)
294 The more programmatic approach for `set_if` could look like this:
298 var srv_vars = service.vars
299 if(len(srv_vars) > 0) {
300 if (typeof(srv_vars.compellent) == Dictionary) {
301 return srv_vars.compellent.contains("disks")
303 log(LogInformationen, "checkcommand set_if", "custom attribute compellent_checks is not a dictionary, ignoring it.")
307 log(LogWarning, "checkcommand set_if", "empty custom attributes")
314 ### <a id="use-functions-command-attribute"></a> Use Functions as Command Attribute
316 This comes in handy for [NotificationCommands](6-object-types.md#objecttype-notificationcommand)
317 or [EventCommands](6-object-types.md#objecttype-eventcommand) which does not require
318 a returned checkresult including state/output.
320 The following example was taken from the community support channels. The requirement was to
321 specify a custom attribute inside the notification apply rule and decide which notification
322 script to call based on that.
324 object User "short-dummy" {
327 object UserGroup "short-dummy-group" {
328 assign where user.name == "short-dummy"
331 apply Notification "mail-admins-short" to Host {
332 import "mail-host-notification"
333 command = "mail-host-notification-test"
334 user_groups = [ "short-dummy-group" ]
336 assign where host.vars.notification.mail
339 The solution is fairly simple: The `command` attribute is implemented as function returning
340 an array required by the caller Icinga 2.
341 The local variable `mailscript` sets the default value for the notification scrip location.
342 If the notification custom attribute `short` is set, it will override the local variable `mailscript`
344 The `mailscript` variable is then used to compute the final notification command array being
347 You can omit the `log()` calls, they only help debugging.
349 object NotificationCommand "mail-host-notification-test" {
350 import "plugin-notification-command"
352 log("command as function")
353 var mailscript = "mail-host-notification-long.sh"
354 if (notification.vars.short) {
355 mailscript = "mail-host-notification-short.sh"
357 log("Running command")
360 var cmd = [ SysconfDir + "/icinga2/scripts/" + mailscript ]
361 log(LogCritical, "me", cmd)
371 ## <a id="access-object-attributes-at-runtime"></a> Access Object Attributes at Runtime
373 The [Object Accessor Functions](20-library-reference.md#object-accessor-functions)
374 can be used to retrieve references to other objects by name.
376 This allows you to access configuration and runtime object attributes. A detailed
377 list can be found [here](6-object-types.md#object-types).
379 Simple cluster example for accessing two host object states and calculating a virtual
380 cluster state and output:
382 object Host "cluster-host-01" {
383 check_command = "dummy"
385 vars.dummy_text = "This host is down."
388 object Host "cluster-host-02" {
389 check_command = "dummy"
391 vars.dummy_text = "This host is up."
394 object Host "cluster" {
395 check_command = "dummy"
396 vars.cluster_nodes = [ "cluster-host-01", "cluster-host-02" ]
398 vars.dummy_state = {{
401 var cluster_nodes = macro("$cluster_nodes$")
403 for (node in cluster_nodes) {
404 if (get_host(node).state > 0) {
411 if (up_count >= down_count) {
412 return 0 //same up as down -> UP
414 return 1 //something is broken
419 var output = "Cluster hosts:\n"
420 var cluster_nodes = macro("$cluster_nodes$")
422 for (node in cluster_nodes) {
423 output += node + ": " + get_host(node).last_check_result.output + "\n"
431 The following example sets time dependent thresholds for the load check based on the current
432 time of the day compared to the defined time period.
434 object TimePeriod "backup" {
435 import "legacy-timeperiod"
438 monday = "02:00-03:00"
439 tuesday = "02:00-03:00"
440 wednesday = "02:00-03:00"
441 thursday = "02:00-03:00"
442 friday = "02:00-03:00"
443 saturday = "02:00-03:00"
444 sunday = "02:00-03:00"
448 object Host "webserver-with-backup" {
449 check_command = "hostalive"
450 address = "127.0.0.1"
453 object Service "webserver-backup-load" {
454 check_command = "load"
455 host_name = "webserver-with-backup"
457 vars.load_wload1 = {{
458 if (get_time_period("backup").is_inside) {
464 vars.load_cload1 = {{
465 if (get_time_period("backup").is_inside) {
474 ## <a id="check-result-freshness"></a> Check Result Freshness
476 In Icinga 2 active check freshness is enabled by default. It is determined by the
477 `check_interval` attribute and no incoming check results in that period of time.
479 threshold = last check execution time + check interval
481 Passive check freshness is calculated from the `check_interval` attribute if set.
483 threshold = last check result time + check interval
485 If the freshness checks are invalid, a new check is executed defined by the
486 `check_command` attribute.
489 ## <a id="check-flapping"></a> Check Flapping
491 The flapping algorithm used in Icinga 2 does not store the past states but
492 calculcates the flapping threshold from a single value based on counters and
493 half-life values. Icinga 2 compares the value with a single flapping threshold
494 configuration attribute named `flapping_threshold`.
496 Flapping detection can be enabled or disabled using the `enable_flapping` attribute.
499 ## <a id="volatile-services"></a> Volatile Services
501 By default all services remain in a non-volatile state. When a problem
502 occurs, the `SOFT` state applies and once `max_check_attempts` attribute
503 is reached with the check counter, a `HARD` state transition happens.
504 Notifications are only triggered by `HARD` state changes and are then
505 re-sent defined by the `interval` attribute.
507 It may be reasonable to have a volatile service which stays in a `HARD`
508 state type if the service stays in a `NOT-OK` state. That way each
509 service recheck will automatically trigger a notification unless the
510 service is acknowledged or in a scheduled downtime.
513 ## <a id="external-commands"></a> External Commands
515 Icinga 2 provides an external command pipe for processing commands
516 triggering specific actions (for example rescheduling a service check
517 through the web interface).
519 In order to enable the `ExternalCommandListener` configuration use the
520 following command and restart Icinga 2 afterwards:
522 # icinga2 feature enable command
524 Icinga 2 creates the command pipe file as `/var/run/icinga2/cmd/icinga2.cmd`
525 using the default configuration.
527 Web interfaces and other Icinga addons are able to send commands to
528 Icinga 2 through the external command pipe, for example for rescheduling
529 a forced service check:
531 # /bin/echo "[`date +%s`] SCHEDULE_FORCED_SVC_CHECK;localhost;ping4;`date +%s`" >> /var/run/icinga2/cmd/icinga2.cmd
533 # tail -f /var/log/messages
535 Oct 17 15:01:25 icinga-server icinga2: Executing external command: [1382014885] SCHEDULE_FORCED_SVC_CHECK;localhost;ping4;1382014885
536 Oct 17 15:01:25 icinga-server icinga2: Rescheduling next check for service 'ping4'
538 A list of currently supported external commands can be found [here](22-appendix.md#external-commands-list-detail).
540 Detailed information on the commands and their required parameters can be found
541 on the [Icinga 1.x documentation](http://docs.icinga.org/latest/en/extcommands2.html).
543 ## <a id="logging"></a> Logging
545 Icinga 2 supports three different types of logging:
548 * Syslog (on *NIX-based operating systems)
549 * Console logging (`STDOUT` on tty)
551 You can enable additional loggers using the `icinga2 feature enable`
552 and `icinga2 feature disable` commands to configure loggers:
554 Feature | Description
555 ---------|------------
556 debuglog | Debug log (path: `/var/log/icinga2/debug.log`, severity: `debug` or higher)
557 mainlog | Main log (path: `/var/log/icinga2/icinga2.log`, severity: `information` or higher)
558 syslog | Syslog (severity: `warning` or higher)
560 By default file the `mainlog` feature is enabled. When running Icinga 2
561 on a terminal log messages with severity `information` or higher are
562 written to the console.
564 ## <a id="performance-data"></a> Performance Data
566 When a host or service check is executed plugins should provide so-called
567 `performance data`. Next to that additional check performance data
568 can be fetched using Icinga 2 runtime macros such as the check latency
569 or the current service state (or additional custom attributes).
571 The performance data can be passed to external applications which aggregate and
572 store them in their backends. These tools usually generate graphs for historical
573 reporting and trending.
575 Well-known addons processing Icinga performance data are PNP4Nagios,
576 inGraph and Graphite.
578 ### <a id="writing-performance-data-files"></a> Writing Performance Data Files
580 PNP4Nagios, inGraph and Graphios use performance data collector daemons to fetch
581 the current performance files for their backend updates.
583 Therefore the Icinga 2 `PerfdataWriter` object allows you to define
584 the output template format for host and services backed with Icinga 2
587 host_format_template = "DATATYPE::HOSTPERFDATA\tTIMET::$icinga.timet$\tHOSTNAME::$host.name$\tHOSTPERFDATA::$host.perfdata$\tHOSTCHECKCOMMAND::$host.check_command$\tHOSTSTATE::$host.state$\tHOSTSTATETYPE::$host.state_type$"
588 service_format_template = "DATATYPE::SERVICEPERFDATA\tTIMET::$icinga.timet$\tHOSTNAME::$host.name$\tSERVICEDESC::$service.name$\tSERVICEPERFDATA::$service.perfdata$\tSERVICECHECKCOMMAND::$service.check_command$\tHOSTSTATE::$host.state$\tHOSTSTATETYPE::$host.state_type$\tSERVICESTATE::$service.state$\tSERVICESTATETYPE::$service.state_type$"
590 The default templates are already provided with the Icinga 2 feature configuration
591 which can be enabled using
593 # icinga2 feature enable perfdata
595 By default all performance data files are rotated in a 15 seconds interval into
596 the `/var/spool/icinga2/perfdata/` directory as `host-perfdata.<timestamp>` and
597 `service-perfdata.<timestamp>`.
598 External collectors need to parse the rotated performance data files and then
599 remove the processed files.
601 ### <a id="graphite-carbon-cache-writer"></a> Graphite Carbon Cache Writer
603 While there are some Graphite collector scripts and daemons like Graphios available for
604 Icinga 1.x it's more reasonable to directly process the check and plugin performance
605 in memory in Icinga 2. Once there are new metrics available, Icinga 2 will directly
606 write them to the defined Graphite Carbon daemon tcp socket.
608 You can enable the feature using
610 # icinga2 feature enable graphite
612 By default the `GraphiteWriter` object expects the Graphite Carbon Cache to listen at
613 `127.0.0.1` on TCP port `2003`.
615 The current naming schema is
617 icinga.<hostname>.<metricname>
618 icinga.<hostname>.<servicename>.<metricname>
620 You can customize the metric prefix name by using the `host_name_template` and
621 `service_name_template` configuration attributes.
623 The example below uses [runtime macros](3-monitoring-basics.md#runtime-macros) and a
624 [global constant](19-language-reference.md#constants) named `GraphiteEnv`. The constant name
625 is freely definable and should be put in the [constants.conf](4-configuring-icinga-2.md#constants-conf) file.
627 const GraphiteEnv = "icinga.env1"
629 object GraphiteWriter "graphite" {
630 host_name_template = GraphiteEnv + ".$host.name$"
631 service_name_template = GraphiteEnv + ".$host.name$.$service.name$"
634 To make sure Icinga 2 writes a valid label into Graphite some characters are replaced
635 with `_` in the target name:
639 The resulting name in Graphite might look like:
641 www-01 / http-cert / response time
642 icinga.www_01.http_cert.response_time
644 In addition to the performance data retrieved from the check plugin, Icinga 2 sends
645 internal check statistic data to Graphite:
648 -------------------|------------------------------------------
649 current_attempt | current check attempt
650 max_check_attempts | maximum check attempts until the hard state is reached
651 reachable | checked object is reachable
652 downtime_depth | number of downtimes this object is in
653 execution_time | check execution time
654 latency | check latency
655 state | current state of the checked object
656 state_type | 0=SOFT, 1=HARD state
658 The following example illustrates how to configure the storage-schemas for Graphite Carbon
659 Cache. Please make sure that the order is correct because the first match wins.
662 pattern = ^icinga\..*\.(max_check_attempts|reachable|current_attempt|execution_time|latency|state|state_type)
666 # intervals like PNP4Nagios uses them per default
668 retentions = 1m:2d,5m:10d,30m:90d,360m:4y
670 ### <a id="gelfwriter"></a> GELF Writer
672 The `Graylog Extended Log Format` (short: [GELF](http://www.graylog2.org/resources/gelf))
673 can be used to send application logs directly to a TCP socket.
675 While it has been specified by the [graylog2](http://www.graylog2.org/) project as their
676 [input resource standard](http://www.graylog2.org/resources/gelf), other tools such as
677 [Logstash](http://www.logstash.net) also support `GELF` as
678 [input type](http://logstash.net/docs/latest/inputs/gelf).
680 You can enable the feature using
682 # icinga2 feature enable gelf
684 By default the `GelfWriter` object expects the GELF receiver to listen at `127.0.0.1` on TCP port `12201`.
685 The default `source` attribute is set to `icinga2`. You can customize that for your needs if required.
687 Currently these events are processed:
692 ### <a id="opentsdb-writer"></a> OpenTSDB Writer
694 While there are some OpenTSDB collector scripts and daemons like tcollector available for
695 Icinga 1.x it's more reasonable to directly process the check and plugin performance
696 in memory in Icinga 2. Once there are new metrics available, Icinga 2 will directly
697 write them to the defined TSDB TCP socket.
699 You can enable the feature using
701 # icinga2 feature enable opentsdb
703 By default the `OpenTsdbWriter` object expects the TSD to listen at
704 `127.0.0.1` on port `4242`.
706 The current naming schema is
708 icinga.host.<metricname>
709 icinga.service.<servicename>.<metricname>
711 for host and service checks. The tag host is always applied.
713 To make sure Icinga 2 writes a valid metric into OpenTSDB some characters are replaced
714 with `_` in the target name:
718 The resulting name in OpenTSDB might look like:
720 www-01 / http-cert / response time
721 icinga.http_cert.response_time
723 In addition to the performance data retrieved from the check plugin, Icinga 2 sends
724 internal check statistic data to OpenTSDB:
727 -------------------|------------------------------------------
728 current_attempt | current check attempt
729 max_check_attempts | maximum check attempts until the hard state is reached
730 reachable | checked object is reachable
731 downtime_depth | number of downtimes this object is in
732 execution_time | check execution time
733 latency | check latency
734 state | current state of the checked object
735 state_type | 0=SOFT, 1=HARD state
737 While reachable, state and state_type are metrics for the host or service the
738 other metrics follow the current naming schema
740 icinga.check.<metricname>
742 with the following tags
745 --------|------------------------------------------
746 type | the check type, one of [host, service]
747 host | hostname, the check ran on
748 service | the service name (if type=service)
752 > You might want to set the tsd.core.auto_create_metrics setting to `true`
753 > in your opentsdb.conf configuration file.
756 ## <a id="status-data"></a> Status Data
758 Icinga 1.x writes object configuration data and status data in a cyclic
759 interval to its `objects.cache` and `status.dat` files. Icinga 2 provides
760 the `StatusDataWriter` object which dumps all configuration objects and
761 status updates in a regular interval.
763 # icinga2 feature enable statusdata
765 Icinga 1.x Classic UI requires this data set as part of its backend.
769 > If you are not using any web interface or addon which uses these files
770 > you can safely disable this feature.
773 ## <a id="compat-logging"></a> Compat Logging
775 The Icinga 1.x log format is considered being the `Compat Log`
776 in Icinga 2 provided with the `CompatLogger` object.
778 These logs are not only used for informational representation in
779 external web interfaces parsing the logs, but also to generate
780 SLA reports and trends in Icinga 1.x Classic UI. Furthermore the
781 [Livestatus](15-livestatus.md#setting-up-livestatus) feature uses these logs for answering queries to
784 The `CompatLogger` object can be enabled with
786 # icinga2 feature enable compatlog
788 By default, the Icinga 1.x log file called `icinga.log` is located
789 in `/var/log/icinga2/compat`. Rotated log files are moved into
790 `var/log/icinga2/compat/archives`.
792 The format cannot be changed without breaking compatibility to
793 existing log parsers.
795 # tail -f /var/log/icinga2/compat/icinga.log
797 [1382115688] LOG ROTATION: HOURLY
798 [1382115688] LOG VERSION: 2.0
799 [1382115688] HOST STATE: CURRENT;localhost;UP;HARD;1;
800 [1382115688] SERVICE STATE: CURRENT;localhost;disk;WARNING;HARD;1;
801 [1382115688] SERVICE STATE: CURRENT;localhost;http;OK;HARD;1;
802 [1382115688] SERVICE STATE: CURRENT;localhost;load;OK;HARD;1;
803 [1382115688] SERVICE STATE: CURRENT;localhost;ping4;OK;HARD;1;
804 [1382115688] SERVICE STATE: CURRENT;localhost;ping6;OK;HARD;1;
805 [1382115688] SERVICE STATE: CURRENT;localhost;processes;WARNING;HARD;1;
806 [1382115688] SERVICE STATE: CURRENT;localhost;ssh;OK;HARD;1;
807 [1382115688] SERVICE STATE: CURRENT;localhost;users;OK;HARD;1;
808 [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;disk;1382115705
809 [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;http;1382115705
810 [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;load;1382115705
811 [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;ping4;1382115705
812 [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;ping6;1382115705
813 [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;processes;1382115705
814 [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;ssh;1382115705
815 [1382115706] EXTERNAL COMMAND: SCHEDULE_FORCED_SVC_CHECK;localhost;users;1382115705
816 [1382115731] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;localhost;ping6;2;critical test|
817 [1382115731] SERVICE ALERT: localhost;ping6;CRITICAL;SOFT;2;critical test
822 ## <a id="db-ido"></a> DB IDO
824 The IDO (Icinga Data Output) modules for Icinga 2 take care of exporting all
825 configuration and status information into a database. The IDO database is used
826 by a number of projects including Icinga Web 1.x and 2.
828 Details on the installation can be found in the [Configuring DB IDO](2-getting-started.md#configuring-db-ido-mysql)
829 chapter. Details on the configuration can be found in the
830 [IdoMysqlConnection](6-object-types.md#objecttype-idomysqlconnection) and
831 [IdoPgsqlConnection](6-object-types.md#objecttype-idopgsqlconnection)
832 object configuration documentation.
833 The DB IDO feature supports [High Availability](12-distributed-monitoring-ha.md#high-availability-db-ido) in
834 the Icinga 2 cluster.
836 The following example query checks the health of the current Icinga 2 instance
837 writing its current status to the DB IDO backend table `icinga_programstatus`
838 every 10 seconds. By default it checks 60 seconds into the past which is a reasonable
839 amount of time - adjust it for your requirements. If the condition is not met,
840 the query returns an empty result.
844 > Use [check plugins](13-addons-plugins.md#plugins) to monitor the backend.
846 Replace the `default` string with your instance name, if different.
850 # mysql -u root -p icinga -e "SELECT status_update_time FROM icinga_programstatus ps
851 JOIN icinga_instances i ON ps.instance_id=i.instance_id
852 WHERE (UNIX_TIMESTAMP(ps.status_update_time) > UNIX_TIMESTAMP(NOW())-60)
853 AND i.instance_name='default';"
855 +---------------------+
856 | status_update_time |
857 +---------------------+
858 | 2014-05-29 14:29:56 |
859 +---------------------+
862 Example for PostgreSQL:
864 # export PGPASSWORD=icinga; psql -U icinga -d icinga -c "SELECT ps.status_update_time FROM icinga_programstatus AS ps
865 JOIN icinga_instances AS i ON ps.instance_id=i.instance_id
866 WHERE ((SELECT extract(epoch from status_update_time) FROM icinga_programstatus) > (SELECT extract(epoch from now())-60))
867 AND i.instance_name='default'";
870 ------------------------
871 2014-05-29 15:11:38+02
875 A detailed list on the available table attributes can be found in the [DB IDO Schema documentation](22-appendix.md#schema-db-ido).
878 ## <a id="check-result-files"></a> Check Result Files
880 Icinga 1.x writes its check result files to a temporary spool directory
881 where they are processed in a regular interval.
882 While this is extremely inefficient in performance regards it has been
883 rendered useful for passing passive check results directly into Icinga 1.x
884 skipping the external command pipe.
886 Several clustered/distributed environments and check-aggregation addons
887 use that method. In order to support step-by-step migration of these
888 environments, Icinga 2 supports the `CheckResultReader` object.
890 There is no feature configuration available, but it must be defined
891 on-demand in your Icinga 2 objects configuration.
893 object CheckResultReader "reader" {
894 spool_dir = "/data/check-results"