Also apply a new structure.
Deleted the old cluster and client documentation and started "fresh".
This commit also includes various images used in the newly written
"Distributed Monitoring" chapter for better understanding.
fixes #12386
* [Register](https://accounts.icinga.org/register) an Icinga account.
* Create a new issue at the [Icinga 2 Development Tracker](https://dev.icinga.org/projects/i2).
-* When reporting a bug, please include the details described in the [Troubleshooting](16-troubleshooting.md#troubleshooting-information-required) chapter (version, configs, logs, etc.).
+* When reporting a bug, please include the details described in the [Troubleshooting](15-troubleshooting.md#troubleshooting-information-required) chapter (version, configs, logs, etc.).
## <a id="whats-new"></a> What's New
These templates are imported by the provided example configuration.
+> **Note**:
+>
+> These templates are built into the binaries. By convention
+> all command objects should import these templates.
+
### <a id="itl-plugin-check-command"></a> plugin-check-command
Command template for check plugins executed by Icinga 2.
### <a id="windows-plugins-load-windows"></a> load-windows
Check command object for the `check_load.exe` plugin.
-This plugin collects the inverse of the performance counter `\Processor(_Total)\% Idle Time` two times, with a wait time of one second between the collection. To change this wait time use [`perfmon-windows`](7-icinga-template-library.md#windows-plugins-load-windows).
+This plugin collects the inverse of the performance counter `\Processor(_Total)\% Idle Time` two times, with a wait time of one second between the collection. To change this wait time use [`perfmon-windows`](10-icinga-template-library.md#windows-plugins-load-windows).
Custom attributes:
### <a id="windows-plugins-network-windows"></a> network-windows
Check command object for the `check_network.exe` plugin.
-Collects the total Bytes inbount and outbound for all interfaces in one second, to itemise interfaces or use a different collection interval use [`perfmon-windows`](7-icinga-template-library.md#windows-plugins-load-windows).
+Collects the total Bytes inbount and outbound for all interfaces in one second, to itemise interfaces or use a different collection interval use [`perfmon-windows`](10-icinga-template-library.md#windows-plugins-load-windows).
Custom attributes:
-## <a id="plugins-contrib"></a> Contributed Plugin Check Commands
+## <a id="plugin-contrib"></a> Contributed Plugin Check Commands
The contributed Plugin Check Commands provides various additional command definitions
contributed by community members.
is set to the path where the user installs custom plugins and can be enabled by
uncommenting the corresponding line in [icinga2.conf](4-configuring-icinga-2.md#icinga2-conf):
- include <plugins-contrib>
+ include <plugin-contrib>
-### <a id="plugins-contrib-databases"></a> Databases
+### <a id="plugin-contrib-databases"></a> Databases
All database plugins go in this category.
-#### <a id="plugins-contrib-command-db2_health"></a> db2_health
+#### <a id="plugin-contrib-command-db2_health"></a> db2_health
The plugin `db2_health` utilises Perl DBD::DB2.
For release tarballs and detailed documentation especially on the different modes and required permissions see [https://labs.consol.de](https://labs.consol.de/nagios/check_db2_health/). For development check [https://github.com](https://github.com/lausser/check_db2_health).
db2_health_env_db2_home | **Required.** Specifies the location of the db2 client libraries as environment variable `DB2_HOME`. Defaults to "/opt/ibm/db2/V10.5".
db2_health_env_db2_version | **Optional.** Specifies the DB2 version as environment variable `DB2_VERSION`.
-#### <a id="plugins-contrib-command-mssql_health"></a> mssql_health
+#### <a id="plugin-contrib-command-mssql_health"></a> mssql_health
The plugin `mssql_health` utilises Perl DBD::Sybase based on FreeTDS to connect to MSSQL databases for monitoring.
For release tarballs, detailed documentation especially on the different modes and scripts for creating a monitoring user see [https://labs.consol.de](https://labs.consol.de/nagios/check_mssql_health/). For development check [https://github.com](https://github.com/lausser/check_mssql_health).
mssql_health_offlineok | **Optional.** Set this to true if offline databases are perfectly ok for you. Defaults to false.
mssql_health_commit | **Optional.** Set this to true to turn on autocommit for the dbd::sybase module. Defaults to false.
-#### <a id="plugins-contrib-command-mysql_health"></a> mysql_health
+#### <a id="plugin-contrib-command-mysql_health"></a> mysql_health
The plugin `mysql_health` utilises Perl DBD::MySQL to connect to MySQL databases for monitoring.
For release tarballs and detailed documentation especially on the different modes and required permissions see [https://labs.consol.de](https://labs.consol.de/nagios/check_mysql_health/). For development check [https://github.com](https://github.com/lausser/check_mysql_health).
mysql_health_statefilesdir | **Optional.** An alternate directory where the plugin can save files."
mysql_health_isvalidtime | **Optional.** Signals the plugin to return OK if now is not a valid check time."
-#### <a id="plugins-contrib-command-oracle_health"></a> oracle_health
+#### <a id="plugin-contrib-command-oracle_health"></a> oracle_health
The plugin `oracle_health` utilises Perl DBD::Oracle based on oracle-instantclient-sdk or sqlplus to connect to Oracle databases for monitoring.
For release tarballs and detailed documentation especially on the different modes and required permissions see [https://labs.consol.de](https://labs.consol.de/nagios/check_oracle_health/). For development check [https://github.com](https://github.com/lausser/check_oracle_health).
ORACLE_HOME | **Required.** Specifies the location of the oracle instant client libraries. Defaults to "/usr/lib/oracle/11.2/client64/lib". Can be overridden by setting "oracle_home".
TNS_ADMIN | **Required.** Specifies the location of the tnsnames.ora including the database connection strings. Defaults to "/etc/icinga2/plugin-configs". Can be overridden by setting "oracle_tns_admin".
-#### <a id="plugins-contrib-command-postgres"></a> postgres
+#### <a id="plugin-contrib-command-postgres"></a> postgres
The plugin `postgres` utilises the psql binary to connect to PostgreSQL databases for monitoring.
For release tarballs and detailed documentation especially the different actions and required persmissions see [https://bucardo.org/wiki/Check_postgres](https://bucardo.org/wiki/Check_postgres). For development check [https://github.com](https://github.com/bucardo/check_postgres).
postgres_valtype | **Optional.** Value type of query result for "custom_query".
postgres_reverse | **Optional.** If "postgres_reverse" is set, warning and critical values are reversed for "custom_query" action.
-#### <a id="plugins-contrib-command-mongodb"></a> mongodb
+#### <a id="plugin-contrib-command-mongodb"></a> mongodb
The plugin `mongodb` utilises Python PyMongo.
For development check [https://github.com](https://github.com/mzupan/nagios-plugin-mongodb).
mongodb_collection | **Optional.** Specify the collection to check
mongodb_sampletime | **Optional.** Time used to sample number of pages faults
-#### <a id="plugins-contrib-command-elasticsearch"></a> elasticsearch
+#### <a id="plugin-contrib-command-elasticsearch"></a> elasticsearch
An [ElasticSearch](https://www.elastic.co/products/elasticsearch) availability
and performance monitoring plugin available for download at [GitHub](https://github.com/anchor/nagios-plugin-elasticsearch).
elasticsearch_prefix | **Optional.** Optional prefix (e.g. 'es') for the ElasticSearch API. Defaults to ''.
elasticsearch_yellowcritical | **Optional.** Instead of issuing a 'warning' for a yellow cluster state, issue a 'critical' alert. Defaults to false.
-#### <a id="plugins-contrib-command-redis"></a> redis
+#### <a id="plugin-contrib-command-redis"></a> redis
The plugin `redis` can measure response time, hitrate, memory utilization, check replication sync and more. It can also test data in a specified key (if necessary, doing average or sum on range).
It is provided by `William Leibzon` at [https://github.com](https://github.com/willixix/naglio-plugins/blob/master/check_redis.pl).
redis_replication_delay | **Optional.** Allows to set threshold on replication delay info.
-### <a id="plugins-contrib-hardware"></a> Hardware
+### <a id="plugin-contrib-hardware"></a> Hardware
This category includes all plugins for various hardware checks.
-#### <a id="plugins-contrib-command-hpasm"></a> hpasm
+#### <a id="plugin-contrib-command-hpasm"></a> hpasm
The plugin [check_hpasm](https://labs.consol.de/de/nagios/check_hpasm/index.html) is a plugin to monitor HP hardware through the HP Insight Agent via SNMP.
hpasm_eval-nics | **Optional.** Check network interfaces (and groups). Try it and report me whyt you think about it. I need to build up some know how on this subject. If you get an error and think, it is not justified for your configuration, please tell me about it. (alwasy send the output of "snmpwalk -On .... 1.3.6.1.4.1.232" and a description how you setup your nics and why it is correct opposed to the plugins error message.
-### <a id="plugins-contrib-icingacli"></a> IcingaCLI
+### <a id="plugin-contrib-icingacli"></a> IcingaCLI
This category includes all plugins using the icingacli provided by Icinga Web 2.
-#### <a id="plugins-contrib-icingacli-businessprocess"></a> Business Process
+#### <a id="plugin-contrib-icingacli-businessprocess"></a> Business Process
This subcommand is provided by the [business process module](https://exchange.icinga.org/icinga/Business+Process) and executed as `icingacli-businessprocess`. The module is hosted by the Icinga project on its [project homepage](https://dev.icinga.org/projects/icingaweb2-module-businessprocess).
icingacli_businessprocess_details | **Optional.** Get details for root cause analyses. Defaults to false.
-### <a id="plugins-contrib-ipmi"></a> IPMI Devices
+### <a id="plugin-contrib-ipmi"></a> IPMI Devices
This category includes all plugins for IPMI devices.
-#### <a id="plugins-contrib-command-ipmi-sensor"></a> ipmi-sensor
+#### <a id="plugin-contrib-command-ipmi-sensor"></a> ipmi-sensor
With the plugin `ipmi-sensor` provided by <a href="https://www.thomas-krenn.com/">Thomas-Krenn.AG</a> you can monitor sensor data for IPMI devices. See https://www.thomas-krenn.com/en/wiki/IPMI_Sensor_Monitoring_Plugin for installation and configuration instructions.
ipmi_show_fru | **Optional.** Print the product serial number if it is available in the IPMI FRU data.
ipmi_no_sel_checking | **Optional.** Turn off system event log checking via ipmi-sel.
-### <a id="plugins-contrib-metrics"></a> Metrics
+### <a id="plugin-contrib-metrics"></a> Metrics
This category includes all plugins for metric-based checks.
-#### <a id="plugin-check-command-graphite"></a> graphite
+#### <a id="plugin-contrib-command-graphite"></a> graphite
Check command object for the [check_graphite](https://github.com/obfuscurity/nagios-scripts) plugin.
graphite_zero_on_error | **Optional.** Return 0 on a graphite 500 error.
graphite_link_graph | **Optional.** Add a link in the plugin output, showing a 24h graph for this metric in graphite.
-### <a id="plugins-contrib-network-components"></a> Network Components
+### <a id="plugin-contrib-network-components"></a> Network Components
This category includes all plugins for various network components like routers, switches and firewalls.
-#### <a id="plugins-contrib-command-interfacetable"></a> interfacetable
+#### <a id="plugin-contrib-command-interfacetable"></a> interfacetable
The plugin `interfacetable` generates a html page containing information about the monitored node and all of its interfaces. The actively developed and maintained version is `interfacetable_v3t` provided by `Yannick Charton` on [http://www.tontonitch.com](http://www.tontonitch.com/tiki/tiki-index.php?page=Nagios+plugins+-+interfacetable_v3t) or [https://github.com](https://github.com/Tontonitch/interfacetable_v3t).
interfacetable_tablesplit | **Optional.** Generate multiple interface tables, one per interface type. Defaults to false.
interfacetable_notype | **Optional.** Remove the interface type for each interface. Defaults to false.
-#### <a id="plugins-contrib-command-iftraffic"></a> iftraffic
+#### <a id="plugin-contrib-command-iftraffic"></a> iftraffic
The plugin [check_iftraffic](https://exchange.icinga.org/exchange/iftraffic)
checks the utilization of a given interface name using the SNMP protocol.
iftraffic_crit | **Optional.** Percent of bandwidth usage necessary to result in critical status (defaults to `98`).
iftraffic_max_counter | **Optional.** Maximum counter value of net devices in kilo/mega/giga/bytes.
-#### <a id="plugins-contrib-command-iftraffic64"></a> iftraffic64
+#### <a id="plugin-contrib-command-iftraffic64"></a> iftraffic64
The plugin [check_iftraffic64](https://exchange.icinga.org/exchange/iftraffic64)
checks the utilization of a given interface name using the SNMP protocol.
iftraffic64_crit | **Optional.** Percent of bandwidth usage necessary to result in critical status (defaults to `98`).
iftraffic64_max_counter | **Optional.** Maximum counter value of net devices in kilo/mega/giga/bytes.
-#### <a id="plugins-contrib-command-interfaces"></a> interfaces
+#### <a id="plugin-contrib-command-interfaces"></a> interfaces
The plugin [check_interfaces](https://www.netways.org/projects/check-interfaces)
Check interfaces and utilization.
interfaces_timeout | **Optional.** Sets the SNMP timeout (in ms).
interfaces_sleep | **Optional.** Sleep between every SNMP query (in ms).
-#### <a id="plugins-contrib-command-nwc_health"></a> nwc_health
+#### <a id="plugin-contrib-command-nwc_health"></a> nwc_health
The plugin [check_nwc_health](https://labs.consol.de/de/nagios/check_nwc_health/index.html)
Check switches, router, there interfaces and utilization.
nwc_health_multiline | **Optional.** Multiline output
-### <a id="plugins-contrib-web"></a> Web
+### <a id="plugin-contrib-web"></a> Web
This category includes all plugins for web-based checks.
-#### <a id="plugin-check-command-webinject"></a> webinject
+#### <a id="plugin-contrib-command-webinject"></a> webinject
Check command object for the [check_webinject](http://www.webinject.org/manual.html) plugin.
webinject_report_type | **Optional.** This setting is used to enable output formatting that is compatible for use with specific external programs. The available values you can set this to are: nagios, mrtg, external and standard.
webinject_testcase_file | **Optional.** When you launch WebInject in console mode, you can optionally supply an argument for a testcase file to run. It will look for this file in the directory that webinject.pl resides in. If no filename is passed from the command line, it will look in config.xml for testcasefile declarations. If no files are specified, it will look for a default file named 'testcases.xml' in the current [webinject] directory. If none of these are found, the engine will stop and give you an error.
-#### <a id="plugin-check-command-jmx4perl"></a> jmx4perl
+#### <a id="plugin-contrib-command-jmx4perl"></a> jmx4perl
The plugin `jmx4perl` utilizes the api provided by the jolokia web application to query java message beans on an application server. It is part of the perl module provided by Roland Huß on [cpan](http://search.cpan.org/~roland/jmx4perl/) including a detailed [documentation](http://search.cpan.org/~roland/jmx4perl/scripts/check_jmx4perl) containing installation tutorial, security advices und usage examples.
jmx4perl_server | **Optional.** Symbolic name of server url to use, which needs to be configured in the configuration file.
jmx4perl_check | **Optional.** Name of a check configuration as defined in the configuration file, use array if you need arguments.
-#### <a id="plugins-contrib-squid"></a> squid
+#### <a id="plugin-contrib-command-squid"></a> squid
Plugin for monitoring [Squid](https://exchange.icinga.org/exchange/check_squid).
squid_timeout | **Optional.** Seconds before plugin times out (default: 15).
-#### <a id="plugins-contrib-nginx_status"></a> nginx_status
+#### <a id="plugin-contrib-command-nginx_status"></a> nginx_status
Plugin for monitoring [nginx_status](https://github.com/regilero/check_nginx_status).
nginx_status_critical | **Optional.** Critical threshold (number of active connections, ReqPerSec or ConnPerSec that will cause a CRITICAL) like '20000,200,300'.
-#### <a id="plugins-contrib-apache_status"></a> apache_status
+#### <a id="plugin-contrib-command-apache_status"></a> apache_status
Plugin for monitoring [apache_status](https://github.com/lbetz/check_apache_status).
apache_status_critical | **Optional.** Critical threshold (number of open slots, busy workers and idle workers that will cause a CRITICAL) like ':10,25,:20'.
-#### <a id="plugins-contrib-kdc"></a> kdc
+#### <a id="plugin-contrib-command-kdc"></a> kdc
Plugin for monitoring [kdc](https://exchange.nagios.org/directory/Plugins/Security/check_kdc/details).
kdc_keytab | **Required** Keytab file containing principal's key.
-#### <a id="plugins-contrib-rbl"></a> rbl
+#### <a id="plugin-contrib-command-rbl"></a> rbl
Plugin for monitoring [rbl](https://github.com/matteocorti/check_rbl)
tbl_timeout | **Optional** Seconds before plugin times out (default: 15).
-### <a id="plugins-contrib-operating-system"></a> Operating System
+### <a id="plugin-contrib-operating-system"></a> Operating System
In this category you can find plugins for gathering information about your operating system or the system beneath like memory usage.
-#### <a id="plugins-contrib-command-mem"></a> mem
+#### <a id="plugin-contrib-command-mem"></a> mem
The plugin `mem` is used for gathering information about memory usage on linux and unix hosts. It is able to count cache memory as free when comparing it to the thresholds. It is provided by `Justin Ellison` on [https://github.com](https://github.com/justintime/nagios-plugins). For more details see the developers blog [http://sysadminsjourney.com](http://sysadminsjourney.com/content/2009/06/04/new-and-improved-checkmempl-nagios-plugin).
mem_warning | **Required.** Specify the warning threshold as number interpreted as percent.
mem_critical | **Required.** Specify the critical threshold as number interpreted as percent.
-#### <a id="plugin-contrib-command-running-kernel"></a> running_kernel
+#### <a id="plugin-contrib-command-running_kernel"></a> running_kernel
Check command object for the `check_running_kernel` plugin
-provided by the `nagios-plugins-contrib` package on Debian.
+provided by the `nagios-plugin-contrib` package on Debian.
Custom attributes:
---------------------------|-------------
running\_kernel\_use\_sudo | Whether to run the plugin with `sudo`. Defaults to false except on Ubuntu where it defaults to true.
-#### <a id="plugins-contrib-command-iostat"></a> iostat
+#### <a id="plugin-contrib-command-iostat"></a> iostat
The plugin [check_iostat](https://github.com/dnsmichi/icinga-plugins/blob/master/scripts/check_iostat) is used to monitor I/O with `iostat` on a linux host. The default thresholds are rather high so you can use a grapher for baselining before setting your own.
yum_installroot | **Optional.** Specifies another installation root directory (for example a chroot).
yum_timeout | **Optional.** Set a timeout in seconds after which the plugin will exit (defaults to 55 seconds).
-### <a id="plugins-contrib-virtualization"></a> Virtualization
+### <a id="plugin-contrib-virtualization"></a> Virtualization
This category includes all plugins for various virtualization technologies.
-#### <a id="plugins-contrib-command-esxi-hardware"></a> esxi_hardware
+#### <a id="plugin-contrib-command-esxi-hardware"></a> esxi_hardware
The plugin `esxi_hardware` is a plugin to monitor hardware of ESXi servers through the vmware api and cim service. It is provided by `Claudio Kuenzler` on [http://www.claudiokuenzler.com](http://www.claudiokuenzler.com/nagios-plugins/check_esxi_hardware.php). For instruction on creating the required local user and workarounds for some hardware types have a look on his homepage.
esxi_hardware_notemp | **Optional.** Do not collect temperature performance data, when **esxi_hardware_perfdata** is set to true. Defaults to false.
esxi_hardware_nofan | **Optional.** Do not collect fan performance data, when **esxi_hardware_perfdata** is set to true. Defaults to false.
-### <a id="plugins-contrib-vmware"></a> VMware
+### <a id="plugin-contrib-vmware"></a> VMware
Check commands for the [check_vmware_esx](https://github.com/BaldMansMojo/check_vmware_esx) plugin.
-#### <a id="plugins-contrib-vmware-esx-dc-volumes"></a> vmware-esx-dc-volumes
+#### <a id="plugin-contrib-vmware-esx-dc-volumes"></a> vmware-esx-dc-volumes
Check command object for the `check_vmware_esx` plugin. Shows all datastore volumes info.
vmware_crit | **Optional.** The critical threshold for volumes. Defaults to "90%".
-#### <a id="plugins-contrib-vmware-esx-dc-runtime-info"></a> vmware-esx-dc-runtime-info
+#### <a id="plugin-contrib-vmware-esx-dc-runtime-info"></a> vmware-esx-dc-runtime-info
Check command object for the `check_vmware_esx` plugin. Shows all runtime info for the datacenter/Vcenter.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-dc-runtime-listvms"></a> vmware-esx-dc-runtime-listvms
+#### <a id="plugin-contrib-vmware-esx-dc-runtime-listvms"></a> vmware-esx-dc-runtime-listvms
Check command object for the `check_vmware_esx` plugin. List of vmware machines and their power state. BEWARE!! In larger environments systems can cause trouble displaying the informations needed due to the mass of data. Use **vmware_alertonly** to avoid this.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-dc-runtime-listhost"></a> vmware-esx-dc-runtime-listhost
+#### <a id="plugin-contrib-vmware-esx-dc-runtime-listhost"></a> vmware-esx-dc-runtime-listhost
Check command object for the `check_vmware_esx` plugin. List of VMware ESX hosts and their power state.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-dc-runtime-listcluster"></a> vmware-esx-dc-runtime-listcluster
+#### <a id="plugin-contrib-vmware-esx-dc-runtime-listcluster"></a> vmware-esx-dc-runtime-listcluster
Check command object for the `check_vmware_esx` plugin. List of VMware clusters and their states.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-dc-runtime-issues"></a> vmware-esx-dc-runtime-issues
+#### <a id="plugin-contrib-vmware-esx-dc-runtime-issues"></a> vmware-esx-dc-runtime-issues
Check command object for the `check_vmware_esx` plugin. All issues for the host.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-dc-runtime-status"></a> vmware-esx-dc-runtime-status
+#### <a id="plugin-contrib-vmware-esx-dc-runtime-status"></a> vmware-esx-dc-runtime-status
Check command object for the `check_vmware_esx` plugin. Overall object status (gray/green/red/yellow).
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-dc-runtime-tools"></a> vmware-esx-dc-runtime-tools
+#### <a id="plugin-contrib-vmware-esx-dc-runtime-tools"></a> vmware-esx-dc-runtime-tools
Check command object for the `check_vmware_esx` plugin. Vmware Tools status.
vmware_openvmtools | **Optional** Prevent CRITICAL state for installed and running Open VM Tools.
-#### <a id="plugins-contrib-vmware-esx-soap-host-check"></a> vmware-esx-soap-host-check
+#### <a id="plugin-contrib-vmware-esx-soap-host-check"></a> vmware-esx-soap-host-check
Check command object for the `check_vmware_esx` plugin. Simple check to verify a successfull connection to VMware SOAP API.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-host-uptime"></a> vmware-esx-soap-host-uptime
+#### <a id="plugin-contrib-vmware-esx-soap-host-uptime"></a> vmware-esx-soap-host-uptime
Check command object for the `check_vmware_esx` plugin. Displays uptime of the VMware host.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-host-cpu"></a> vmware-esx-soap-host-cpu
+#### <a id="plugin-contrib-vmware-esx-soap-host-cpu"></a> vmware-esx-soap-host-cpu
Check command object for the `check_vmware_esx` plugin. CPU usage in percentage.
vmware_crit | **Optional.** The critical threshold in percent. Defaults to "90%".
-#### <a id="plugins-contrib-vmware-esx-soap-host-cpu-ready"></a> vmware-esx-soap-host-cpu-ready
+#### <a id="plugin-contrib-vmware-esx-soap-host-cpu-ready"></a> vmware-esx-soap-host-cpu-ready
Check command object for the `check_vmware_esx` plugin. Percentage of time that the virtual machine was ready, but could not get scheduled to run on the physical CPU. CPU ready time is dependent on the number of virtual machines on the host and their CPU loads. High or growing ready time can be a hint CPU bottlenecks.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-host-cpu-wait"></a> vmware-esx-soap-host-cpu-wait
+#### <a id="plugin-contrib-vmware-esx-soap-host-cpu-wait"></a> vmware-esx-soap-host-cpu-wait
Check command object for the `check_vmware_esx` plugin. CPU time spent in wait state. The wait total includes time spent the CPU idle, CPU swap wait, and CPU I/O wait states. High or growing wait time can be a hint I/O bottlenecks.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-host-cpu-usage"></a> vmware-esx-soap-host-cpu-usage
+#### <a id="plugin-contrib-vmware-esx-soap-host-cpu-usage"></a> vmware-esx-soap-host-cpu-usage
Check command object for the `check_vmware_esx` plugin. Actively used CPU of the host, as a percentage of the total available CPU. Active CPU is approximately equal to the ratio of the used CPU to the available CPU.
vmware_crit | **Optional.** The critical threshold in percent. Defaults to "90%".
-#### <a id="plugins-contrib-vmware-esx-soap-host-mem"></a> vmware-esx-soap-host-mem
+#### <a id="plugin-contrib-vmware-esx-soap-host-mem"></a> vmware-esx-soap-host-mem
Check command object for the `check_vmware_esx` plugin. All mem info(except overall and no thresholds).
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-host-mem-usage"></a> vmware-esx-soap-host-mem-usage
+#### <a id="plugin-contrib-vmware-esx-soap-host-mem-usage"></a> vmware-esx-soap-host-mem-usage
Check command object for the `check_vmware_esx` plugin. Average mem usage in percentage.
vmware_crit | **Optional.** The critical threshold in percent. Defaults to "90%".
-#### <a id="plugins-contrib-vmware-esx-soap-host-mem-consumed"></a> vmware-esx-soap-host-mem-consumed
+#### <a id="plugin-contrib-vmware-esx-soap-host-mem-consumed"></a> vmware-esx-soap-host-mem-consumed
Check command object for the `check_vmware_esx` plugin. Amount of machine memory used on the host. Consumed memory includes Includes memory used by the Service Console, the VMkernel vSphere services, plus the total consumed metrics for all running virtual machines in MB.
vmware_crit | **Optional.** The critical threshold in percent. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-mem-swapused"></a> vmware-esx-soap-host-mem-swapused
+#### <a id="plugin-contrib-vmware-esx-soap-host-mem-swapused"></a> vmware-esx-soap-host-mem-swapused
Check command object for the `check_vmware_esx` plugin. Amount of memory that is used by swap. Sum of memory swapped of all powered on VMs and vSphere services on the host in MB. In case of an error all VMs with their swap used will be displayed.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-mem-overhead"></a> vmware-esx-soap-host-mem-overhead
+#### <a id="plugin-contrib-vmware-esx-soap-host-mem-overhead"></a> vmware-esx-soap-host-mem-overhead
Check command object for the `check_vmware_esx` plugin. Additional mem used by VM Server in MB.
vmware_crit | **Optional.** The critical threshold in percent. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-mem-memctl"></a> vmware-esx-soap-host-mem-memctl
+#### <a id="plugin-contrib-vmware-esx-soap-host-mem-memctl"></a> vmware-esx-soap-host-mem-memctl
Check command object for the `check_vmware_esx` plugin. The sum of all vmmemctl values in MB for all powered-on virtual machines, plus vSphere services on the host. If the balloon target value is greater than the balloon value, the VMkernel inflates the balloon, causing more virtual machine memory to be reclaimed. If the balloon target value is less than the balloon value, the VMkernel deflates the balloon, which allows the virtual machine to consume additional memory if needed (used by VM memory control driver). In case of an error all VMs with their vmmemctl values will be displayed.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-net"></a> vmware-esx-soap-host-net
+#### <a id="plugin-contrib-vmware-esx-soap-host-net"></a> vmware-esx-soap-host-net
Check command object for the `check_vmware_esx` plugin. Shows net info.
vmware_isregexp | **Optional.** Treat blacklist expression as regexp.
-#### <a id="plugins-contrib-vmware-esx-soap-host-net-usage"></a> vmware-esx-soap-host-net-usage
+#### <a id="plugin-contrib-vmware-esx-soap-host-net-usage"></a> vmware-esx-soap-host-net-usage
Check command object for the `check_vmware_esx` plugin. Overall network usage in KBps(Kilobytes per Second).
vmware_crit | **Optional.** The critical threshold in KBps(Kilobytes per Second). No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-net-receive"></a> vmware-esx-soap-host-net-receive
+#### <a id="plugin-contrib-vmware-esx-soap-host-net-receive"></a> vmware-esx-soap-host-net-receive
Check command object for the `check_vmware_esx` plugin. Data receive in KBps(Kilobytes per Second).
vmware_crit | **Optional.** The critical threshold in KBps(Kilobytes per Second). No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-net-send"></a> vmware-esx-soap-host-net-send
+#### <a id="plugin-contrib-vmware-esx-soap-host-net-send"></a> vmware-esx-soap-host-net-send
Check command object for the `check_vmware_esx` plugin. Data send in KBps(Kilobytes per Second).
vmware_crit | **Optional.** The critical threshold in KBps(Kilobytes per Second). No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-net-nic"></a> vmware-esx-soap-host-net-nic
+#### <a id="plugin-contrib-vmware-esx-soap-host-net-nic"></a> vmware-esx-soap-host-net-nic
Check command object for the `check_vmware_esx` plugin. Check all active NICs.
vmware_isregexp | **Optional.** Treat blacklist expression as regexp.
-#### <a id="plugins-contrib-vmware-esx-soap-host-volumes"></a> vmware-esx-soap-host-volumes
+#### <a id="plugin-contrib-vmware-esx-soap-host-volumes"></a> vmware-esx-soap-host-volumes
Check command object for the `check_vmware_esx` plugin. Shows all datastore volumes info.
vmware_spaceleft | **Optional.** This has to be used in conjunction with thresholds as mentioned above.
-#### <a id="plugins-contrib-vmware-esx-soap-host-io"></a> vmware-esx-soap-host-io
+#### <a id="plugin-contrib-vmware-esx-soap-host-io"></a> vmware-esx-soap-host-io
Check command object for the `check_vmware_esx` plugin. Shows all disk io info. Without subselect no thresholds can be given. All I/O values are aggregated from historical intervals over the past 24 hours with a 5 minute sample rate.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-host-io-aborted"></a> vmware-esx-soap-host-io-aborted
+#### <a id="plugin-contrib-vmware-esx-soap-host-io-aborted"></a> vmware-esx-soap-host-io-aborted
Check command object for the `check_vmware_esx` plugin. Number of aborted SCSI commands.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-io-resets"></a> vmware-esx-soap-host-io-resets
+#### <a id="plugin-contrib-vmware-esx-soap-host-io-resets"></a> vmware-esx-soap-host-io-resets
Check command object for the `check_vmware_esx` plugin. Number of SCSI bus resets.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-io-read"></a> vmware-esx-soap-host-io-read
+#### <a id="plugin-contrib-vmware-esx-soap-host-io-read"></a> vmware-esx-soap-host-io-read
Check command object for the `check_vmware_esx` plugin. Average number of kilobytes read from the disk each second.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-io-read-latency"></a> vmware-esx-soap-host-io-read-latency
+#### <a id="plugin-contrib-vmware-esx-soap-host-io-read-latency"></a> vmware-esx-soap-host-io-read-latency
Check command object for the `check_vmware_esx` plugin. Average amount of time (ms) to process a SCSI read command issued from the Guest OS to the virtual machine.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-io-write"></a> vmware-esx-soap-host-io-write
+#### <a id="plugin-contrib-vmware-esx-soap-host-io-write"></a> vmware-esx-soap-host-io-write
Check command object for the `check_vmware_esx` plugin. Average number of kilobytes written to disk each second.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-io-write-latency"></a> vmware-esx-soap-host-io-write-latency
+#### <a id="plugin-contrib-vmware-esx-soap-host-io-write-latency"></a> vmware-esx-soap-host-io-write-latency
Check command object for the `check_vmware_esx` plugin. Average amount of time (ms) taken to process a SCSI write command issued by the Guest OS to the virtual machine.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-io-usage"></a> vmware-esx-soap-host-io-usage
+#### <a id="plugin-contrib-vmware-esx-soap-host-io-usage"></a> vmware-esx-soap-host-io-usage
Check command object for the `check_vmware_esx` plugin. Aggregated disk I/O rate. For hosts, this metric includes the rates for all virtual machines running on the host.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-io-kernel-latency"></a> vmware-esx-soap-host-io-kernel-latency
+#### <a id="plugin-contrib-vmware-esx-soap-host-io-kernel-latency"></a> vmware-esx-soap-host-io-kernel-latency
Check command object for the `check_vmware_esx` plugin. Average amount of time (ms) spent by VMkernel processing each SCSI command.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-io-device-latency"></a> vmware-esx-soap-host-io-device-latency
+#### <a id="plugin-contrib-vmware-esx-soap-host-io-device-latency"></a> vmware-esx-soap-host-io-device-latency
Check command object for the `check_vmware_esx` plugin. Average amount of time (ms) to complete a SCSI command from the physical device.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-io-queue-latency"></a> vmware-esx-soap-host-io-queue-latency
+#### <a id="plugin-contrib-vmware-esx-soap-host-io-queue-latency"></a> vmware-esx-soap-host-io-queue-latency
Check command object for the `check_vmware_esx` plugin. Average amount of time (ms) spent in the VMkernel queue.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-io-total-latency"></a> vmware-esx-soap-host-io-total-latency
+#### <a id="plugin-contrib-vmware-esx-soap-host-io-total-latency"></a> vmware-esx-soap-host-io-total-latency
Check command object for the `check_vmware_esx` plugin. Average amount of time (ms) taken during the collection interval to process a SCSI command issued by the guest OS to the virtual machine. The sum of kernelWriteLatency and deviceWriteLatency.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-media"></a> vmware-esx-soap-host-media
+#### <a id="plugin-contrib-vmware-esx-soap-host-media"></a> vmware-esx-soap-host-media
Check command object for the `check_vmware_esx` plugin. List vm's with attached host mounted media like cd,dvd or floppy drives. This is important for monitoring because a virtual machine with a mount cd or dvd drive can not be moved to another host.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-service"></a> vmware-esx-soap-host-service
+#### <a id="plugin-contrib-vmware-esx-soap-host-service"></a> vmware-esx-soap-host-service
Check command object for the `check_vmware_esx` plugin. Shows host service info.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-runtime"></a> vmware-esx-soap-host-runtime
+#### <a id="plugin-contrib-vmware-esx-soap-host-runtime"></a> vmware-esx-soap-host-runtime
Check command object for the `check_vmware_esx` plugin. Shows runtime info: VMs, overall status, connection state, health, storagehealth, temperature and sensor are represented as one value and without thresholds.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-host-runtime-con"></a> vmware-esx-soap-host-runtime-con
+#### <a id="plugin-contrib-vmware-esx-soap-host-runtime-con"></a> vmware-esx-soap-host-runtime-con
Check command object for the `check_vmware_esx` plugin. Shows connection state.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-host-runtime-listvms"></a> vmware-esx-soap-host-runtime-listvms
+#### <a id="plugin-contrib-vmware-esx-soap-host-runtime-listvms"></a> vmware-esx-soap-host-runtime-listvms
Check command object for the `check_vmware_esx` plugin. List of VMware machines and their status.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-runtime-status"></a> vmware-esx-soap-host-runtime-status
+#### <a id="plugin-contrib-vmware-esx-soap-host-runtime-status"></a> vmware-esx-soap-host-runtime-status
Check command object for the `check_vmware_esx` plugin. Overall object status (gray/green/red/yellow).
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-host-runtime-health"></a> vmware-esx-soap-host-runtime-health
+#### <a id="plugin-contrib-vmware-esx-soap-host-runtime-health"></a> vmware-esx-soap-host-runtime-health
Check command object for the `check_vmware_esx` plugin. Checks cpu/storage/memory/sensor status.
vmware_isregexp | **Optional.** Treat blacklist and whitelist expressions as regexp.
-#### <a id="plugins-contrib-vmware-esx-soap-host-runtime-health-listsensors"></a> vmware-esx-soap-host-runtime-health-listsensors
+#### <a id="plugin-contrib-vmware-esx-soap-host-runtime-health-listsensors"></a> vmware-esx-soap-host-runtime-health-listsensors
Check command object for the `check_vmware_esx` plugin. List all available sensors(use for listing purpose only).
vmware_isregexp | **Optional.** Treat blacklist and whitelist expressions as regexp.
-#### <a id="plugins-contrib-vmware-esx-soap-host-runtime-health-nostoragestatus"></a> vmware-esx-soap-host-runtime-health-nostoragestatus
+#### <a id="plugin-contrib-vmware-esx-soap-host-runtime-health-nostoragestatus"></a> vmware-esx-soap-host-runtime-health-nostoragestatus
Check command object for the `check_vmware_esx` plugin. This is to avoid a double alarm if you use **vmware-esx-soap-host-runtime-health** and **vmware-esx-soap-host-runtime-storagehealth**.
vmware_isregexp | **Optional.** Treat blacklist and whitelist expressions as regexp.
-#### <a id="plugins-contrib-vmware-esx-soap-host-runtime-storagehealth"></a> vmware-esx-soap-host-runtime-storagehealth
+#### <a id="plugin-contrib-vmware-esx-soap-host-runtime-storagehealth"></a> vmware-esx-soap-host-runtime-storagehealth
Check command object for the `check_vmware_esx` plugin. Local storage status check.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-runtime-temp"></a> vmware-esx-soap-host-runtime-temp
+#### <a id="plugin-contrib-vmware-esx-soap-host-runtime-temp"></a> vmware-esx-soap-host-runtime-temp
Check command object for the `check_vmware_esx` plugin. Lists all temperature sensors.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-runtime-issues"></a> vmware-esx-soap-host-runtime-issues
+#### <a id="plugin-contrib-vmware-esx-soap-host-runtime-issues"></a> vmware-esx-soap-host-runtime-issues
Check command object for the `check_vmware_esx` plugin. Lists all configuration issues for the host.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-storage"></a> vmware-esx-soap-host-storage
+#### <a id="plugin-contrib-vmware-esx-soap-host-storage"></a> vmware-esx-soap-host-storage
Check command object for the `check_vmware_esx` plugin. Shows Host storage info.
vmware_isregexp | **Optional.** Treat blacklist and whitelist expressions as regexp.
-#### <a id="plugins-contrib-vmware-esx-soap-host-storage-adapter"></a> vmware-esx-soap-host-storage-adapter
+#### <a id="plugin-contrib-vmware-esx-soap-host-storage-adapter"></a> vmware-esx-soap-host-storage-adapter
Check command object for the `check_vmware_esx` plugin. List host bus adapters.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-storage-lun"></a> vmware-esx-soap-host-storage-lun
+#### <a id="plugin-contrib-vmware-esx-soap-host-storage-lun"></a> vmware-esx-soap-host-storage-lun
Check command object for the `check_vmware_esx` plugin. List SCSI logical units. The listing will include: LUN, canonical name of the disc, all of displayed name which is not part of the canonical name and status.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-host-storage-path"></a> vmware-esx-soap-host-storage-path
+#### <a id="plugin-contrib-vmware-esx-soap-host-storage-path"></a> vmware-esx-soap-host-storage-path
Check command object for the `check_vmware_esx` plugin. List multipaths and the associated paths.
vmware_multiline | **Optional.** Multiline output in overview. This mean technically that a multiline output uses a HTML **\<br\>** for the GUI. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-vm-cpu"></a> vmware-esx-soap-vm-cpu
+#### <a id="plugin-contrib-vmware-esx-soap-vm-cpu"></a> vmware-esx-soap-vm-cpu
Check command object for the `check_vmware_esx` plugin. Shows all CPU usage info.
-#### <a id="plugins-contrib-vmware-esx-soap-vm-cpu-ready"></a> vmware-esx-soap-vm-cpu-ready
+#### <a id="plugin-contrib-vmware-esx-soap-vm-cpu-ready"></a> vmware-esx-soap-vm-cpu-ready
Check command object for the `check_vmware_esx` plugin. Percentage of time that the virtual machine was ready, but could not get scheduled to run on the physical CPU.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-vm-cpu-wait"></a> vmware-esx-soap-vm-cpu-wait
+#### <a id="plugin-contrib-vmware-esx-soap-vm-cpu-wait"></a> vmware-esx-soap-vm-cpu-wait
Check command object for the `check_vmware_esx` plugin. CPU time spent in wait state. The wait total includes time spent the CPU idle, CPU swap wait, and CPU I/O wait states. High or growing wait time can be a hint I/O bottlenecks.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-vm-cpu-usage"></a> vmware-esx-soap-vm-cpu-usage
+#### <a id="plugin-contrib-vmware-esx-soap-vm-cpu-usage"></a> vmware-esx-soap-vm-cpu-usage
Check command object for the `check_vmware_esx` plugin. Amount of actively used virtual CPU, as a percentage of total available CPU. This is the host's view of the CPU usage, not the guest operating system view. It is the average CPU utilization over all available virtual CPUs in the virtual machine.
vmware_crit | **Optional.** Critical threshold in percent. Defaults to "90%".
-#### <a id="plugins-contrib-vmware-esx-soap-vm-mem"></a> vmware-esx-soap-vm-mem
+#### <a id="plugin-contrib-vmware-esx-soap-vm-mem"></a> vmware-esx-soap-vm-mem
Check command object for the `check_vmware_esx` plugin. Shows all memory info, except overall.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-vm-mem-usage"></a> vmware-esx-soap-vm-mem-usage
+#### <a id="plugin-contrib-vmware-esx-soap-vm-mem-usage"></a> vmware-esx-soap-vm-mem-usage
Check command object for the `check_vmware_esx` plugin. Average mem usage in percentage of configured virtual machine "physical" memory.
vmware_crit | **Optional.** Critical threshold in percent. Defaults to "90%".
-#### <a id="plugins-contrib-vmware-esx-soap-vm-mem-consumed"></a> vmware-esx-soap-vm-mem-consumed
+#### <a id="plugin-contrib-vmware-esx-soap-vm-mem-consumed"></a> vmware-esx-soap-vm-mem-consumed
Check command object for the `check_vmware_esx` plugin. Amount of guest physical memory in MB consumed by the virtual machine for guest memory. Consumed memory does not include overhead memory. It includes shared memory and memory that might be reserved, but not actually used. Use this metric for charge-back purposes.<br>
**vm consumed memory = memory granted -- memory saved**
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-vm-mem-memctl"></a> vmware-esx-soap-vm-mem-memctl
+#### <a id="plugin-contrib-vmware-esx-soap-vm-mem-memctl"></a> vmware-esx-soap-vm-mem-memctl
Check command object for the `check_vmware_esx` plugin. Amount of guest physical memory that is currently reclaimed from the virtual machine through ballooning. This is the amount of guest physical memory that has been allocated and pinned by the balloon driver.
-#### <a id="plugins-contrib-vmware-esx-soap-vm-net"></a> vmware-esx-soap-vm-net
+#### <a id="plugin-contrib-vmware-esx-soap-vm-net"></a> vmware-esx-soap-vm-net
Check command object for the `check_vmware_esx` plugin. Shows net info.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-vm-net-usage"></a> vmware-esx-soap-vm-net-usage
+#### <a id="plugin-contrib-vmware-esx-soap-vm-net-usage"></a> vmware-esx-soap-vm-net-usage
Check command object for the `check_vmware_esx` plugin. Overall network usage in KBps(Kilobytes per Second).
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-vm-net-receive"></a> vmware-esx-soap-vm-net-receive
+#### <a id="plugin-contrib-vmware-esx-soap-vm-net-receive"></a> vmware-esx-soap-vm-net-receive
Check command object for the `check_vmware_esx` plugin. Receive in KBps(Kilobytes per Second).
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-vm-net-send"></a> vmware-esx-soap-vm-net-send
+#### <a id="plugin-contrib-vmware-esx-soap-vm-net-send"></a> vmware-esx-soap-vm-net-send
Check command object for the `check_vmware_esx` plugin. Send in KBps(Kilobytes per Second).
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-vm-io"></a> vmware-esx-soap-vm-io
+#### <a id="plugin-contrib-vmware-esx-soap-vm-io"></a> vmware-esx-soap-vm-io
Check command object for the `check_vmware_esx` plugin. SShows all disk io info. Without subselect no thresholds can be given. All I/O values are aggregated from historical intervals over the past 24 hours with a 5 minute sample rate.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-vm-io-read"></a> vmware-esx-soap-vm-io-read
+#### <a id="plugin-contrib-vmware-esx-soap-vm-io-read"></a> vmware-esx-soap-vm-io-read
Check command object for the `check_vmware_esx` plugin. Average number of kilobytes read from the disk each second.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-vm-io-write"></a> vmware-esx-soap-vm-io-write
+#### <a id="plugin-contrib-vmware-esx-soap-vm-io-write"></a> vmware-esx-soap-vm-io-write
Check command object for the `check_vmware_esx` plugin. Average number of kilobytes written to disk each second.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-vm-io-usage"></a> vmware-esx-soap-vm-io-usage
+#### <a id="plugin-contrib-vmware-esx-soap-vm-io-usage"></a> vmware-esx-soap-vm-io-usage
Check command object for the `check_vmware_esx` plugin. Aggregated disk I/O rate.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-vm-runtime"></a> vmware-esx-soap-vm-runtime
+#### <a id="plugin-contrib-vmware-esx-soap-vm-runtime"></a> vmware-esx-soap-vm-runtime
Check command object for the `check_vmware_esx` plugin. Shows virtual machine runtime info.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-vm-runtime-con"></a> vmware-esx-soap-vm-runtime-con
+#### <a id="plugin-contrib-vmware-esx-soap-vm-runtime-con"></a> vmware-esx-soap-vm-runtime-con
Check command object for the `check_vmware_esx` plugin. Shows the connection state.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-vm-runtime-powerstate"></a> vmware-esx-soap-vm-runtime-powerstate
+#### <a id="plugin-contrib-vmware-esx-soap-vm-runtime-powerstate"></a> vmware-esx-soap-vm-runtime-powerstate
Check command object for the `check_vmware_esx` plugin. Shows virtual machine power state: poweredOn, poweredOff or suspended.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-vm-runtime-status"></a> vmware-esx-soap-vm-runtime-status
+#### <a id="plugin-contrib-vmware-esx-soap-vm-runtime-status"></a> vmware-esx-soap-vm-runtime-status
Check command object for the `check_vmware_esx` plugin. Overall object status (gray/green/red/yellow).
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-vm-runtime-consoleconnections"></a> vmware-esx-soap-vm-runtime-consoleconnections
+#### <a id="plugin-contrib-vmware-esx-soap-vm-runtime-consoleconnections"></a> vmware-esx-soap-vm-runtime-consoleconnections
Check command object for the `check_vmware_esx` plugin. Console connections to virtual machine.
vmware_crit | **Optional.** The critical threshold. No value defined as default.
-#### <a id="plugins-contrib-vmware-esx-soap-vm-runtime-gueststate"></a> vmware-esx-soap-vm-runtime-gueststate
+#### <a id="plugin-contrib-vmware-esx-soap-vm-runtime-gueststate"></a> vmware-esx-soap-vm-runtime-gueststate
Check command object for the `check_vmware_esx` plugin. Guest OS status. Needs VMware Tools installed and running.
vmware_password | **Optional.** The username's password. No value defined as default.
vmware_authfile | **Optional.** Use auth file instead username/password to session connect. No effect if **vmware_username** and **vmware_password** are defined <br> **Autentication file content:** <br> username=vmuser <br> password=p@ssw0rd
-#### <a id="plugins-contrib-vmware-esx-soap-vm-runtime-tools"></a> vmware-esx-soap-vm-runtime-tools
+#### <a id="plugin-contrib-vmware-esx-soap-vm-runtime-tools"></a> vmware-esx-soap-vm-runtime-tools
Check command object for the `check_vmware_esx` plugin. Guest OS status. VMware tools status.
vmware_openvmtools | **Optional** Prevent CRITICAL state for installed and running Open VM Tools.
-#### <a id="plugins-contrib-vmware-esx-soap-vm-runtime-issues"></a> vmware-esx-soap-vm-runtime-issues
+#### <a id="plugin-contrib-vmware-esx-soap-vm-runtime-issues"></a> vmware-esx-soap-vm-runtime-issues
Check command object for the `check_vmware_esx` plugin. All issues for the virtual machine.
+++ /dev/null
-# <a id="monitoring-remote-systems"></a> Monitoring Remote Systems
-
-## <a id="monitoring-remote-systems-overview"></a> Overview
-
-There's a variety of possibilities to monitor remote servers and services. First off you should
-decide how your primary monitoring master is able to reach these hosts and services.
-
-* direct connection querying the service interface (for example `http`), so-called [agent-less checks](10-monitoring-remote-systems.md#agent-less-checks)
-* local checks requiring an additional daemon as communication device for your monitoring server
-
-## <a id="agent-less-checks"></a> Agent-less Checks
-
-If the remote service is available using a network protocol and port,
-and a [check plugin](2-getting-started.md#setting-up-check-plugins) is available, you don't
-necessarily need a local client installed. Rather choose a plugin and
-configure all parameters and thresholds. The [Icinga 2 Template Library](7-icinga-template-library.md#icinga-template-library)
-already ships various examples like
-
-* [ping4](7-icinga-template-library.md#plugin-check-command-ping4), [ping6](7-icinga-template-library.md#plugin-check-command-ping6),
-[fping4](7-icinga-template-library.md#plugin-check-command-fping4), [fping6](7-icinga-template-library.md#plugin-check-command-fping6), [hostalive](7-icinga-template-library.md#plugin-check-command-hostalive)
-* [tcp](7-icinga-template-library.md#plugin-check-command-tcp), [udp](7-icinga-template-library.md#plugin-check-command-udp), [ssl](7-icinga-template-library.md#plugin-check-command-ssl)
-* [http](7-icinga-template-library.md#plugin-check-command-http), [ftp](7-icinga-template-library.md#plugin-check-command-ftp)
-* [smtp](7-icinga-template-library.md#plugin-check-command-smtp), [ssmtp](7-icinga-template-library.md#plugin-check-command-ssmtp),
-[imap](7-icinga-template-library.md#plugin-check-command-imap), [simap](7-icinga-template-library.md#plugin-check-command-simap),
-[pop](7-icinga-template-library.md#plugin-check-command-pop), [spop](7-icinga-template-library.md#plugin-check-command-spop)
-* [ntp_time](7-icinga-template-library.md#plugin-check-command-ntp-time)
-* [ssh](7-icinga-template-library.md#plugin-check-command-ssh)
-* [dns](7-icinga-template-library.md#plugin-check-command-dns), [dig](7-icinga-template-library.md#plugin-check-command-dig), [dhcp](7-icinga-template-library.md#plugin-check-command-dhcp)
-
-There are numerous check plugins contributed by community members available
-on the internet. If you found one for your requirements, [integrate them into Icinga 2](3-monitoring-basics.md#command-plugin-integration).
-
-Start your search at
-
-* [Icinga Exchange](https://exchange.icinga.org)
-* [Icinga Wiki](https://wiki.icinga.org)
-
-An example is provided in the sample configuration in the getting started
-section provided by Icinga 2 ([hosts.conf](4-configuring-icinga-2.md#hosts-conf), [services.conf](4-configuring-icinga-2.md#services-conf)).
-
-
-## <a id="agent-based-checks"></a> Agent-based Checks
-
-If the remote services are not directly accessible through the network, a
-local agent installation exposing the results to check queries can
-become handy.
-
-Icinga 2 itself can be used as agent (client, satellite) in this scenario, but there
-are also a couple of addons available for this task.
-
-The most famous ones are listed below.
-
-## <a id="agent-based-checks-linux-unix"></a> Agent-based Checks for Linux/Unix
-
-The agent runs as daemon and communicates with the master requesting a check being executed
-or local stored information (SNMP OID). The Icinga 2 client continues to execute checks
-when the connection dies, and does not need the master as check scheduler like the other
-listed agents.
-
-* Icinga 2 Client
-* SSH
-* SNMP
-* NRPE
-
-## <a id="agent-based-checks-windows"></a> Agent-based Checks for Windows
-
-The Windows agent runs as administrative service and offers direct plugin execution and/or
-local check result being sent to the master instance.
-
-* Icinga 2 Client
-* NSClient++
-
-SNMP could also be used, but was deprecated in Windows Server 2012. Alternatively you can
-look into the WMI interface.
-
options.
# icinga2
- icinga2 - The Icinga 2 network monitoring daemon (version: v2.1.1-299-gf695275)
+ icinga2 - The Icinga 2 network monitoring daemon (version: v2.5.0)
Usage:
icinga2 <command> [<arguments>]
### Libraries
-Instead of loading libraries using the [`library` config directive](18-language-reference.md#library)
+Instead of loading libraries using the [`library` config directive](17-language-reference.md#library)
you can also use the `--library` command-line option.
### Constants
-[Global constants](18-language-reference.md#constants) can be set using the `--define` command-line option.
+[Global constants](17-language-reference.md#constants) can be set using the `--define` command-line option.
### <a id="config-include-path"></a> Config Include Path
## <a id="cli-command-console"></a> CLI command: Console
The CLI command `console` can be used to evaluate Icinga config expressions, e.g. to test
-[functions](18-language-reference.md#functions).
+[functions](17-language-reference.md#functions).
$ icinga2 console
Icinga 2 (version: v2.4.0)
$ rlwrap icinga2 console
The `console` can be used to connect to a running Icinga 2 instance using
-the [REST API](9-icinga2-api.md#icinga2-api). [API permissions](9-icinga2-api.md#icinga2-api-permissions)
+the [REST API](12-icinga2-api.md#icinga2-api). [API permissions](12-icinga2-api.md#icinga2-api-permissions)
are required for executing config expressions and auto-completion.
> **Note**
## <a id="cli-command-daemon"></a> CLI command: Daemon
The CLI command `daemon` provides the functionality to start/stop Icinga 2.
-Furthermore it provides the [configuration validation](8-cli-commands.md#config-validation).
+Furthermore it provides the [configuration validation](11-cli-commands.md#config-validation).
# icinga2 daemon --help
- icinga2 - The Icinga 2 network monitoring daemon (version: v2.1.1-299-gf695275)
+ icinga2 - The Icinga 2 network monitoring daemon (version: v2.5.0)
Usage:
icinga2 daemon [<arguments>]
The `--validate` option can be used to check if your configuration files
contain errors. If any errors are found, the exit status is 1, otherwise 0
-is returned. More details in the [configuration validation](8-cli-commands.md#config-validation) chapter.
+is returned. More details in the [configuration validation](11-cli-commands.md#config-validation) chapter.
## <a id="cli-command-feature"></a> CLI command: Feature
## <a id="cli-command-node"></a> CLI command: Node
Provides the functionality to install and manage master and client
-nodes in a [remote monitoring ](11-icinga2-client.md#icinga2-client) or
-[distributed cluster](13-distributed-monitoring-ha.md#distributed-monitoring-high-availability) scenario.
+nodes in a [distributed monitoring](6-distributed-monitoring.md#distributed-monitoring) scenario.
# icinga2 node --help
- icinga2 - The Icinga 2 network monitoring daemon (version: v2.1.1-299-gf695275)
+ icinga2 - The Icinga 2 network monitoring daemon (version: v2.5.0)
Usage:
icinga2 <command> [<arguments>]
The `object` CLI command can be used to list all configuration objects and their
attributes. The command also shows where each of the attributes was modified.
-That way you can also identify which objects have been created from your [apply rules](18-language-reference.md#apply).
+That way you can also identify which objects have been created from your [apply rules](17-language-reference.md#apply).
-More information can be found in the [troubleshooting](16-troubleshooting.md#list-configuration-objects) section.
+More information can be found in the [troubleshooting](15-troubleshooting.md#list-configuration-objects) section.
# icinga2 object --help
- icinga2 - The Icinga 2 network monitoring daemon (version: v2.1.1-299-gf695275)
+ icinga2 - The Icinga 2 network monitoring daemon (version: v2.5.0)
Usage:
icinga2 <command> [<arguments>]
* request a signed certificate from the master
* generate a new ticket for the client setup
-This functionality is used by the [node setup/wizard](8-cli-commands.md#cli-command-pki) CLI commands too.
+This functionality is used by the [node setup/wizard](11-cli-commands.md#cli-command-pki) CLI commands too.
# icinga2 pki --help
- icinga2 - The Icinga 2 network monitoring daemon (version: v2.1.1-299-gf695275)
+ icinga2 - The Icinga 2 network monitoring daemon (version: v2.5.0)
Usage:
icinga2 <command> [<arguments>]
## <a id="cli-command-repository"></a> CLI command: Repository
-Provides the functionality to manage the Icinga 2 configuration repository in
-`/etc/icinga2/repository.d`. All changes are logged and must be committed or
-cleared after review.
-
-
-> **Note**
->
-> The CLI command `repository` only supports basic configuration manipulation (add, remove)
-> and a limited set of objects required for the [remote client] integration. Future
-> versions will support more options (set, etc.).
->
-> Please check the Icinga 2 development roadmap for updates.
-
-
- # icinga2 repository --help
- icinga2 - The Icinga 2 network monitoring daemon (version: v2.1.1-299-gf695275)
-
- Usage:
- icinga2 <command> [<arguments>]
-
- Supported commands:
- * repository clear-changes (clear uncommitted repository changes)
- * repository commit (commit repository changes)
- * repository endpoint add (adds a new Endpoint object)
- * repository endpoint list (lists all Endpoint objects)
- * repository endpoint remove (removes a Endpoint object)
- * repository host add (adds a new Host object)
- * repository host list (lists all Host objects)
- * repository host remove (removes a Host object)
- * repository service add (adds a new Service object)
- * repository service list (lists all Service objects)
- * repository service remove (removes a Service object)
- * repository zone add (adds a new Zone object)
- * repository zone list (lists all Zone objects)
- * repository zone remove (removes a Zone object)
-
- Global options:
- -h [ --help ] show this help message
- -V [ --version ] show version information
- --color use VT100 color codes even when stdout is not a
- terminal
- -D [ --define ] arg define a constant
- -a [ --app ] arg application library name (default: icinga)
- -l [ --library ] arg load a library
- -I [ --include ] arg add include search directory
- -x [ --log-level ] arg specify the log level for the console log
-
- Command options:
-
- Report bugs at <https://dev.icinga.org/>
- Icinga home page: <https://www.icinga.org/>
-
-
+This command is not supported anymore. Parts of its functionality
+are used in the [node update-config](11-cli-commands.md#cli-command-node) cli command.
## <a id="cli-command-troubleshoot"></a> CLI command: Troubleshoot
-Collects basic information like version, paths, log files and crash reports for troubleshooting purposes and prints them to a file or the console. See [troubleshooting](16-troubleshooting.md#troubleshooting-information-required).
+Collects basic information like version, paths, log files and crash reports for troubleshooting purposes and prints them to a file or the console. See [troubleshooting](15-troubleshooting.md#troubleshooting-information-required).
Its output defaults to a file named `troubleshooting-[TIMESTAMP].log` so it won't overwrite older troubleshooting files.
## <a id="cli-command-variable"></a> CLI command: Variable
-Lists all configured variables (constants) in a similar fasion like [object list](8-cli-commands.md#cli-command-object).
+Lists all configured variables (constants) in a similar fasion like [object list](11-cli-commands.md#cli-command-object).
# icinga2 variable --help
- icinga2 - The Icinga 2 network monitoring daemon (version: v2.1.1-299-gf695275)
+ icinga2 - The Icinga 2 network monitoring daemon (version: v2.5.0)
Usage:
icinga2 <command> [<arguments>]
Icinga 2 provides configuration files for some commonly used features. These
are installed in the `/etc/icinga2/features-available` directory and can be
enabled and disabled using the `icinga2 feature enable` and `icinga2 feature disable`
-[CLI commands](8-cli-commands.md#cli-command-feature), respectively.
+[CLI commands](11-cli-commands.md#cli-command-feature), respectively.
The `icinga2 feature enable` CLI command creates symlinks in the
`/etc/icinga2/features-enabled` directory which is included by default
> `# icinga2 daemon -C`
If you encounter errors during configuration validation, please make sure
-to read the [troubleshooting](16-troubleshooting.md#troubleshooting) chapter.
+to read the [troubleshooting](15-troubleshooting.md#troubleshooting) chapter.
-You can also use the [CLI command](8-cli-commands.md#cli-command-object) `icinga2 object list`
+You can also use the [CLI command](11-cli-commands.md#cli-command-object) `icinga2 object list`
after validation passes to analyze object attributes, inheritance or created
objects by apply rules.
-Find more on troubleshooting with `object list` in [this chapter](16-troubleshooting.md#list-configuration-objects).
+Find more on troubleshooting with `object list` in [this chapter](15-troubleshooting.md#list-configuration-objects).
Example filtered by `Service` objects with the name `ping*`:
## <a id="config-change-reload"></a> Reload on Configuration Changes
Everytime you have changed your configuration you should first tell Icinga 2
-to [validate](8-cli-commands.md#config-validation). If there are no validation errors, you can
+to [validate](11-cli-commands.md#config-validation). If there are no validation errors, you can
safely reload the Icinga 2 daemon.
# /etc/init.d/icinga2 reload
> which will validate the configuration in a separate process and not stop
> the other events like check execution, notifications, etc.
>
-> Details can be found [here](23-migrating-from-icinga-1x.md#differences-1x-2-real-reload).
+> Details can be found [here](22-migrating-from-icinga-1x.md#differences-1x-2-real-reload).
+++ /dev/null
-# <a id="icinga2-client"></a> Icinga 2 Client
-
-## <a id="icinga2-client-introduction"></a> Introduction
-
-Icinga 2 uses its own unique and secure communitication protol amongst instances.
-Be it an High-Availability cluster setup, distributed load-balanced setup or just a single
-agent [monitoring a remote client](11-icinga2-client.md#icinga2-client).
-
-All communication is secured by TLS with certificates, and fully supports IPv4 and IPv6.
-
-If you are planning to use the native Icinga 2 cluster feature for distributed
-monitoring and high-availability, please continue reading in
-[this chapter](distributed-monitoring-high-availability).
-
-> **Tip**
->
-> Don't panic -- there are CLI commands available, including setup wizards for easy installation
-> with SSL certificates.
-> If you prefer to use your own CA (for example Puppet), you can do that as well.
-
-
-## <a id="icinga2-client-scenarios"></a> Client Scenarios
-
-* Clients with [local configuration](11-icinga2-client.md#icinga2-client-configuration-local), sending their inventory to the master
-* Clients as [command execution bridge](11-icinga2-client.md#icinga2-client-configuration-command-bridge) without local configuration
-* Clients receive their configuration from the master using the [cluster config sync](11-icinga2-client.md#icinga2-client-configuration-master-config-sync)
-
-Keep the [naming convention](13-distributed-monitoring-ha.md#cluster-naming-convention) for nodes in mind.
-
-> **Tip**
->
-> If you're looking for troubleshooting clients problems, check the general
-> [cluster troubleshooting](16-troubleshooting.md#troubleshooting-cluster) section.
-
-### <a id="icinga2-client-configuration-combined-scenarios"></a> Combined Client Scenarios
-
-If your setup consists of remote clients with local configuration but also command execution bridges
-and probably syncing global templates through the cluster config sync, you should take a deep
-breath and take pen and paper to draw your design before starting over.
-
-Keep the following hints in mind:
-
-* You can blacklist remote nodes entirely. They are then ignored on `node update-config`
-on the master.
-* Your remote instance can have local configuration **and** act as remote command execution bridge.
-* You can use the `global` cluster zones to sync check commands, templates, etc. to your remote clients.
-Be it just for command execution or for helping the local configuration.
-* If your remote clients shouldn't have local configuration, remove `conf.d` inclusion from `icinga2`
-and simply use the cluster configuration sync.
-* `accept_config` and `accept_commands` are disabled by default in the `api` feature
-
-If you are planning to use the Icinga 2 client inside a distributed setup, refer to
-[this chapter](13-distributed-monitoring-ha.md#cluster-scenarios-master-satellite-clients) with detailed instructions.
-
-
-## <a id="icinga2-client-installation"></a> Installation
-
-### <a id="icinga2-client-installation-firewall"></a> Configure the Firewall
-
-Icinga 2 master, satellite and client instances communicate using the default tcp
-port `5665`.
-
-Communication between zones requires one of these connection directions:
-
-* The parent zone nodes are able to connect to the child zone nodes (`parent => child`).
-* The child zone nodes are able to connect to the parent zone nodes (`parent <= child`).
-* Both connnection directions work.
-
-If you are going to use CSR-Autosigning, you must (temporarly) allow the client
-connecting to the master instance and open the firewall port. Once the client install is done,
-you can close the port and use a different communication direction (master-to-client).
-
-In case of a [multiple hierarchy setup](13-distributed-monitoring-ha.md#cluster-scenarios-master-satellite-clients)
-(master, satellite, client) you will need to manually deploy your [client certificates](11-icinga2-client.md#certificates-manual-creation)
-and zone configuration.
-
-### <a id="icinga2-client-installation-master-setup"></a> Setup the Master for Remote Clients
-
-If you are planning to use the [remote Icinga 2 clients](11-icinga2-client.md#icinga2-client),
-you'll first need to update your master setup.
-
-Your master setup requires the following
-
-* SSL CA and signed certificate for the master
-* Enabled API feature, and a local Endpoint and Zone object configuration
-* Firewall ACLs for the communication port (default 5665)
-
-You can use the [CLI command](8-cli-commands.md#cli-command-node) `node wizard` for setting up a new node
-on the master. The command must be run as root, all Icinga 2 specific files
-will be updated to the icinga user the daemon is running as (certificate files
-for example).
-
-Make sure to answer the first question with `n` (no).
-
- nbmif /etc/icinga2 # icinga2 node wizard
- Welcome to the Icinga 2 Setup Wizard!
-
- We'll guide you through all required configuration details.
-
- Please specify if this is a satellite setup ('n' installs a master setup) [Y/n]: n
- Starting the Master setup routine...
- Please specifiy the common name (CN) [icinga2-node1.localdomain]:
- Checking the 'api' feature...
- 'api' feature not enabled, running 'api setup' now.
- information/cli: Generating new CA.
-
- information/base: Writing private key to '/var/lib/icinga2/ca/ca.key'.
- information/base: Writing X509 certificate to '/var/lib/icinga2/ca/ca.crt'.
- information/cli: Initializing serial file in '/var/lib/icinga2/ca/serial.txt'.
- information/cli: Generating new CSR in '/etc/icinga2/pki/icinga2-node1.localdomain.csr'.
-
- information/base: Writing private key to '/etc/icinga2/pki/icinga2-node1.localdomain.key'.
- information/base: Writing certificate signing request to '/etc/icinga2/pki/icinga2-node1.localdomain.csr'.
- information/cli: Signing CSR with CA and writing certificate to '/etc/icinga2/pki/icinga2-node1.localdomain.crt'.
-
- information/cli: Copying CA certificate to '/etc/icinga2/pki/ca.crt'.
-
- information/cli: Adding new ApiUser 'root' in '/etc/icinga2/conf.d/api-users.conf'.
-
- information/cli: Enabling the ApiListener feature.
-
- Enabling feature api. Make sure to restart Icinga 2 for these changes to take effect.
- information/cli: Dumping config items to file '/etc/icinga2/zones.conf'.
- Please specify the API bind host/port (optional):
- Bind Host []:
- Bind Port []:
- information/cli: Updating constants.conf.
- information/cli: Updating constants file '/etc/icinga2/constants.conf'.
- information/cli: Updating constants file '/etc/icinga2/constants.conf'.
- Done.
-
- Now restart your Icinga 2 daemon to finish the installation!
-
-
-The setup wizard will do the following:
-
-* Check if the `api` feature is already enabled, and if not:
- * Generate a local CA in `/var/lib/icinga2/ca` or use the existing one
- * Generate a new CSR, sign it with the local CA and copying it into `/etc/icinga2/pki`
- * Enabling the API feature, and setting optional `bind_host` and `bind_port`
-* Generate a local zone and endpoint configuration for this master based on FQDN
-* Setting the `NodeName` and `TicketSalt` constants in [constants.conf](4-configuring-icinga-2.md#constants-conf)
-
-The setup wizard does not automatically restart Icinga 2.
-
-Verify the modified configuration:
-
- # egrep 'NodeName|TicketSalt' /etc/icinga2/constants.conf
-
- # cat /etc/icinga2/zones.conf
- /*
- * Generated by Icinga 2 node setup commands
- * on 2015-02-09 15:21:49 +0100
- */
-
- object Endpoint "icinga2-node1.localdomain" {
- }
-
- object Zone "master" {
- //this is the local node master named = "master"
- endpoints = [ "icinga2-node1.localdomain" ]
- }
-
-Validate the configuration and restart Icinga 2.
-
-
-> **Note**
->
-> This setup wizard will install a standalone master, HA cluster scenarios are currently
-> not supported and require manual modifications afterwards.
-
-## <a id="icinga2-client-setup"></a> Client Setup for Remote Monitoring
-
-Icinga 2 can be installed on Linux/Unix and Windows. While
-[Linux/Unix](11-icinga2-client.md#icinga2-client-installation-client-setup-linux) will be using the [CLI command](8-cli-commands.md#cli-command-node)
-`node wizard` for a guided setup, you will need to use the
-graphical installer for Windows based client setup.
-
-Your client setup requires the following
-
-* A ready configured and installed [master node](11-icinga2-client.md#icinga2-client-installation-master-setup)
-* SSL signed certificate for communication with the master (Use [CSR auto-signing](11-icinga2-client.md#csr-autosigning-requirements)).
-* Enabled API feature, and a local Endpoint and Zone object configuration
-* Firewall ACLs for the communication port (default 5665)
-
-
-
-### <a id="csr-autosigning-requirements"></a> Requirements for CSR Auto-Signing
-
-If your remote clients are capable of connecting to the central master, Icinga 2
-supports CSR auto-signing.
-
-First you'll need to define a secure ticket salt in the [constants.conf](4-configuring-icinga-2.md#constants-conf).
-The [setup wizard for the master setup](11-icinga2-client.md#icinga2-client-installation-master-setup) will create
-one for you already.
-
-> **Note**
->
-> **Never** expose the ticket salt to your clients. This is the master's private key
-> and must remain on the master providing the CSR Auto-Signing functionality for security reasons.
-
-The client setup wizard will ask you to generate a valid ticket number using its CN.
-If you already know your remote client's Common Names (CNs) -- usually the FQDN --, you
-can generate all ticket numbers on-demand.
-
-This is also reasonable if you are not capable of installing the remote client, but
-a colleague of yours, or a customer.
-
-Example for a client:
-
- # icinga2 pki ticket --cn icinga2-node2.localdomain
-
-
-### <a id="certificates-manual-creation"></a> Manual SSL Certificate Generation
-
-This is described separately in the [cluster setup chapter](13-distributed-monitoring-ha.md#manual-certificate-generation).
-
-> **Note**
->
-> If you're using [CSR Auto-Signing](11-icinga2-client.md#csr-autosigning-requirements), skip this step.
-
-
-### <a id="icinga2-client-installation-client-setup-linux"></a> Setup the Client on Linux
-
-There is no extra client binary or package required. Install Icinga 2 from your distribution's package
-repository as described in the general [installation instructions](2-getting-started.md#setting-up-icinga2).
-
-Please make sure that either [CSR Auto-Signing](11-icinga2-client.md#csr-autosigning-requirements) requirements
-are fulfilled, or that you're using [manual SSL certificate generation](13-distributed-monitoring-ha.md#manual-certificate-generation).
-
-> **Note**
->
-> You don't need any features (DB IDO, Livestatus) or user interfaces on the remote client.
-> Install them only if you're planning to use them.
-
-Once the package installation succeeded, use the `node wizard` CLI command to install
-a new Icinga 2 node as client setup.
-
-You'll need the following configuration details:
-
-* The client common name (CN). Defaults to FQDN.
-* The client's local zone name. Defaults to FQDN.
-* The master endpoint name. Look into your master setup `zones.conf` file for the proper name.
-* The master endpoint connection information. Your master's IP address and port (port defaults to 5665)
-* The [request ticket number](11-icinga2-client.md#csr-autosigning-requirements) generated on your master
-for CSR Auto-Signing
-* Bind host/port for the Api feature (optional)
-
-The command must be run as root, all Icinga 2 specific files will be updated to the icinga
-user the daemon is running as (certificate files for example). The wizard creates backups
-of configuration and certificate files if already existing.
-
-Capitalized options in square brackets (e.g. `[Y/n]`) signal the default value and
-allow you to continue pressing `Enter` instead of entering a value.
-
- # icinga2 node wizard
- Welcome to the Icinga 2 Setup Wizard!
- We'll guide you through all required configuration details.
-
- Please specify if this is a satellite setup ('n' installs a master setup) [Y/n]:
- Starting the Node setup routine...
- Please specifiy the common name (CN) [icinga2-node2.localdomain]:
- Please specifiy the local zone name [icinga2-node2.localdomain]:
- Please specify the master endpoint(s) this node should connect to:
- Master Common Name (CN from your master setup): icinga2-node1.localdomain
- Please fill out the master connection information:
- Master endpoint host (optional, your master's IP address or FQDN): 192.168.56.101
- Master endpoint port (optional) []:
- Add more master endpoints? [y/N]
- Please specify the master connection for CSR auto-signing (defaults to master endpoint host):
- Host [192.168.56.101]:
- Port [5665]:
- information/base: Writing private key to '/etc/icinga2/pki/icinga2-node2.localdomain.key'.
- information/base: Writing X509 certificate to '/etc/icinga2/pki/icinga2-node2.localdomain.crt'.
- information/cli: Generating self-signed certifiate:
- information/cli: Fetching public certificate from master (192.168.56.101, 5665):
-
- information/cli: Writing trusted certificate to file '/etc/icinga2/pki/trusted-master.crt'.
- information/cli: Stored trusted master certificate in '/etc/icinga2/pki/trusted-master.crt'.
-
- Please specify the request ticket generated on your Icinga 2 master.
- (Hint: # icinga2 pki ticket --cn 'icinga2-node2.localdomain'): ead2d570e18c78abf285d6b85524970a0f69c22d
- information/cli: Processing self-signed certificate request. Ticket 'ead2d570e18c78abf285d6b85524970a0f69c22d'.
-
- information/cli: Writing signed certificate to file '/etc/icinga2/pki/icinga2-node2.localdomain.crt'.
- information/cli: Writing CA certificate to file '/etc/icinga2/pki/ca.crt'.
- Please specify the API bind host/port (optional):
- Bind Host []:
- Bind Port []:
- information/cli: Disabling the Notification feature.
- Disabling feature notification. Make sure to restart Icinga 2 for these changes to take effect.
- information/cli: Enabling the Apilistener feature.
- Enabling feature api. Make sure to restart Icinga 2 for these changes to take effect.
- information/cli: Created backup file '/etc/icinga2/features-available/api.conf.orig'.
- information/cli: Generating local zones.conf.
- information/cli: Dumping config items to file '/etc/icinga2/zones.conf'.
- information/cli: Created backup file '/etc/icinga2/zones.conf.orig'.
- information/cli: Updating constants.conf.
- information/cli: Created backup file '/etc/icinga2/constants.conf.orig'.
- information/cli: Updating constants file '/etc/icinga2/constants.conf'.
- information/cli: Updating constants file '/etc/icinga2/constants.conf'.
- Done.
-
- Now restart your Icinga 2 daemon to finish the installation!
-
-
-The setup wizard will do the following:
-
-* Generate a new self-signed certificate and copy it into `/etc/icinga2/pki`
-* Store the master's certificate as trusted certificate for requesting a new signed certificate
-(manual step when using `node setup`).
-* Request a new signed certificate from the master and store updated certificate and master CA in `/etc/icinga2/pki`
-* Generate a local zone and endpoint configuration for this client and the provided master information
-(based on FQDN)
-* Disabling the `notification` feature for this client
-* Enabling the `api` feature, and setting optional `bind_host` and `bind_port`
-* Setting the `NodeName` constant in [constants.conf](4-configuring-icinga-2.md#constants-conf)
-
-The setup wizard does not automatically restart Icinga 2.
-
-Verify the modified configuration:
-
- # grep 'NodeName' /etc/icinga2/constants.conf
-
- # cat /etc/icinga2/zones.conf
- /*
- * Generated by Icinga 2 node setup commands
- * on 2015-02-09 16:56:10 +0100
- */
-
- object Endpoint "icinga2-node1.localdomain" {
- host = "192.168.56.101"
- }
-
- object Zone "master" {
- endpoints = [ "icinga2-node1.localdomain" ]
- }
-
- object Endpoint "icinga2-node2.localdomain" {
- }
-
- object Zone "icinga2-node2.localdomain" {
- //this is the local node = "icinga2-node2.localdomain"
- endpoints = [ "icinga2-node2.localdomain" ]
- parent = "master"
- }
-
-Validate the configuration and restart Icinga 2.
-
-If you are getting an error when requesting the ticket number, please check the following:
-
-* Can your client connect to the master instance?
-* Is the CN the same (from pki ticket on the master and setup node on the client)?
-* Is the ticket expired?
-
-
-
-#### <a id="icinga2-client-installation-client-setup-linux-manual"></a> Manual Setup without Wizard
-
-Instead of using the `node wizard` cli command, there is an alternative `node setup`
-cli command available which has some pre-requisites. Make sure that the
-`/etc/icinga2/pki` exists and is owned by the `icinga` user (or the user Icinga 2 is
-running as).
-
-`icinga2-node1.localdomain` is the already installed master instance while
-`icinga2-node2.localdomain` is the instance where the installation cli commands
-are executed.
-
-Required information:
-
-* The client common name (CN). Use the FQDN, e.g. `icinga2-node2.localdomain`.
-* The master host and zone name. Pass that to `pki save-cert` as `--host` parameter for example.
- * Optional: Master endpoint host and port information for the `--endpoint` parameter.
-* The client ticket number generated on the master (`icinga2 pki ticket --cn icinga2-node2.localdomain`)
-
-Generate a new local self-signed certificate.
-
- # icinga2 pki new-cert --cn icinga2-node2.localdomain \
- --key /etc/icinga2/pki/icinga2-node2.localdomain.key \
- --cert /etc/icinga2/pki/icinga2-node2.localdomain.crt
-
-Request the master certificate from the master host (`icinga2-node1.localdomain`)
-and store it as `trusted-master.crt`. Review it and continue.
-
- # icinga2 pki save-cert --key /etc/icinga2/pki/icinga2-node2.localdomain.key \
- --cert /etc/icinga2/pki/icinga2-node2.localdomain.crt \
- --trustedcert /etc/icinga2/pki/trusted-master.crt \
- --host icinga2-node1.localdomain
-
-Send the self-signed certificate to the master host using the ticket number and
-receive a CA signed certificate and the master's `ca.crt` certificate.
-Specify the path to the previously stored trusted master certificate.
-
- # icinga2 pki request --host icinga2-node1.localdomain \
- --port 5665 \
- --ticket ead2d570e18c78abf285d6b85524970a0f69c22d \
- --key /etc/icinga2/pki/icinga2-node2.localdomain.key \
- --cert /etc/icinga2/pki/icinga2-node2.localdomain.crt \
- --trustedcert /etc/icinga2/pki/trusted-master.crt \
- --ca /etc/icinga2/pki/ca.crt
-
-Continue with the additional node setup steps. Specify a local endpoint and zone name (`icinga2-node2.localdomain`)
-and set the master host (`icinga2-node1.localdomain`) as parent zone configuration. Specify the path to
-the previously stored trusted master certificate.
-
- # icinga2 node setup --ticket ead2d570e18c78abf285d6b85524970a0f69c22d \
- --endpoint icinga2-node1.localdomain \
- --zone icinga2-node2.localdomain \
- --master_host icinga2-node1.localdomain \
- --trustedcert /etc/icinga2/pki/trusted-master.crt
-
-In case the client should connect to the master node, you'll
-need to modify the `--endpoint` parameter using the format `cn,host,port`.
-
- --endpoint icinga2-node1.localdomain,192.168.56.101,5665
-
-Restart Icinga 2 once complete.
-
- # service icinga2 restart
-
-
-### <a id="icinga2-client-installation-client-setup-windows"></a> Setup the Client on Windows
-
-Download the MSI-Installer package from [http://packages.icinga.org/windows/](http://packages.icinga.org/windows/).
-
-Requirements:
-
-* Windows Vista/Server 2008 or higher
-* [Microsoft .NET Framework 2.0](http://www.microsoft.com/de-de/download/details.aspx?id=1639) if not already installed.
-
-The installer package includes the [NSClient++](http://www.nsclient.org/) so Icinga 2 can
-use its built-in plugins. You can use the [nscp-local commands from the ITL](7-icinga-template-library.md#nscp-plugin-check-commands)
-for these plugins.
-
-
-
-
-
-
-
-The graphical installer will offer to run the Icinga 2 setup wizard after the installation.
-You can also manually run the Icinga 2 setup wizard from the start menu.
-
-On a fresh installation the setup wizard will guide you through the initial configuration
-as well as the required details for SSL certificate generation using CSR-Autosigning.
-
-You'll need the following configuration details:
-
-* The client common name (CN). Defaults to FQDN.
-* The [request ticket number](11-icinga2-client.md#csr-autosigning-requirements) generated on your master
-for CSR Auto-Signing
-
-Example on the master:
-
- icinga2 pki ticket --cn DESKTOP-IHRPO96
-
-Fill in the required information and click `Add` to add a new master connection.
-
-
-
-Add the following details:
-
-* The master endpoint name. Look into your master setup `zones.conf` file for the proper name.
-* The master endpoint connection information. Your master's IP address and port (defaults to 5665)
-
-
-
-You can optionally enable the following settings:
-
-* Accept commands from master (client as [command execution bridge](11-icinga2-client.md#icinga2-client-configuration-command-bridge)).
-* Accept config updates from master ([client receiving configuration](11-icinga2-client.md#icinga2-client-scenarios))
-* Install/Update NSClient++
-
-
-
-The next step allows you to verify the CA presented by the master.
-
-
-
-If you have chosen to install/update the NSClient++ package, the Icinga 2 setup wizard will ask
-you to do so.
-
-
-
-Finish the setup wizard.
-
-
-
-Once install and configuration is done, Icinga 2 is automatically started as a Windows service.
-
-
-
-The Icinga 2 configuration is located inside the `C:\ProgramData\icinga2` directory.
-If you click `Examine Config` in the setup wizard, it will open a new explorer window.
-
-
-
-The configuration files can be modified with your favorite editor.
-
-Configuration validation is done similar to the linux pendant on the Windows shell:
-
- C:> icinga2.exe daemon -C
-
-**Note**: You have to run this command in a shell with `administrator` permissions.
-
-In case you want to restart the Icinga 2 service, run `services.msc` and restart the
-`icinga2` service. Alternatively you can use the `net {start,stop}` CLI commands.
-
-#### <a id="icinga2-client-installation-client-setup-windows-silent"></a> Silent Windows Client Setup
-
-If you want to install the client silently/unattended, use the `/qn` modifier. The
-installation should not trigger a restart but if you want to be completly sure you can use the `/norestart` modifier.
-
- C:> msiexec /i C:\Icinga2-v2.4.5-x86.msi /qn /norestart
-
-## <a id="icinga2-client-configuration-modes"></a> Client Configuration Modes
-
-* Clients with [local configuration](11-icinga2-client.md#icinga2-client-configuration-local), sending their inventory to the master
-* Clients as [command execution bridge](11-icinga2-client.md#icinga2-client-configuration-command-bridge) without local configuration
-* Clients receive their configuration from the master ([Cluster config sync](11-icinga2-client.md#icinga2-client-configuration-master-config-sync))
-
-### <a id="icinga2-client-configuration-local"></a> Clients with Local Configuration
-
-This is considered as independant satellite using a local scheduler, configuration
-and the possibility to add Icinga 2 features on demand.
-
-There is no difference in the configuration syntax on clients to any other Icinga 2 installation.
-You can also use additional features like notifications directly on the remote client if you are
-required to. Basically everything a single Icinga 2 instance provides by default.
-
-The following convention applies to remote clients:
-
-* The hostname in the default host object should be the same as the Common Name (CN) used for SSL setup
-* Add new services and check commands locally
-
-Local configured checks are transferred to the central master. There are additional `node`
-cli commands available which allow you to list/add/remove/blacklist remote clients and
-generate the configuration on the master.
-
-
-#### <a id="icinga2-remote-monitoring-master-discovery"></a> Discover Client Services on the Master
-
-Icinga 2 clients will sync their locally defined objects to the defined master node. That way you can
-list, add, filter and remove nodes based on their `node`, `zone`, `host` or `service` name.
-
-List all discovered nodes (satellites, agents) and their hosts/services:
-
- # icinga2 node list
- Node 'icinga2-node2.localdomain' (last seen: Mon Feb 9 16:58:21 2015)
- * Host 'icinga2-node2.localdomain'
- * Service 'ping4'
- * Service 'ping6'
- * Service 'ssh'
- * Service 'http'
- * Service 'disk'
- * Service 'disk /'
- * Service 'icinga'
- * Service 'load'
- * Service 'procs'
- * Service 'swap'
- * Service 'users'
-
-Listing the node and its host(s) and service(s) does not modify the master configuration yet. You
-need to generate the configuration in the next step.
-
-
-### <a id="icinga2-client-master-discovery-generate-config"></a> Generate Configuration for Client Services on the Master
-
-There is a dedicated Icinga 2 CLI command for updating the client services on the master,
-generating all required configuration.
-
- # icinga2 node update-config
-
-The generated configuration of all nodes is stored in the `repository.d/` directory.
-
-By default, the following additional configuration is generated:
-* add `Endpoint` and `Zone` objects for the newly added node
-* add `cluster-zone` health check for the master host for reachability and dependencies
-* use the default templates `satellite-host` and `satellite-service` defined in `/etc/icinga2/conf.d/satellite.conf`
-* apply a dependency for all other hosts on the remote satellite prevening failure checks/notifications
-
-If hosts or services disappeared from the client discovery, it will remove the existing configuration objects
-from the config repository. If there are existing hosts/services defined or modified, the CLI command will not
-overwrite these (modified) configuration files.
-
-After updating the configuration repository, make sure to reload Icinga 2.
-
- # service icinga2 reload
-
-Using systemd:
-
- # systemctl reload icinga2
-
-
-The `update-config` CLI command will fail if there are uncommitted changes for the
-configuration repository or if your master is part of a HA setup (see https://dev.icinga.org/issues/8292 for details).
-Please review these changes manually, or clear the commit and try again. This is a
-safety hook to prevent unwanted manual changes to be committed by a updating the
-client discovered objects only.
-
- # icinga2 repository commit --simulate
- # icinga2 repository clear-changes
- # icinga2 repository commit
-
-
-### <a id="icinga2-client-configuration-command-bridge"></a> Clients as Command Execution Bridge
-
-Similar to other addons (NRPE, NSClient++, etc.) the remote Icinga 2 client will only
-execute commands the master instance is sending. There are no local host or service
-objects configured, only the check command definitions must be configured.
-
-> **Note**
->
-> Remote clients must explicitely accept commands in a similar
-> fashion as cluster nodes [accept configuration](13-distributed-monitoring-ha.md#cluster-zone-config-sync).
-> This is due to security reasons.
-
-Edit the `api` feature configuration in `/etc/icinga2/features-enabled/api.conf` on your client
-and set `accept_commands` to `true`.
-
- object ApiListener "api" {
- cert_path = SysconfDir + "/icinga2/pki/" + NodeName + ".crt"
- key_path = SysconfDir + "/icinga2/pki/" + NodeName + ".key"
- ca_path = SysconfDir + "/icinga2/pki/ca.crt"
- accept_commands = true
- }
-
-Icinga 2 on the remote client does not schedule checks locally, or keep checking
-hosts/services on connection loss. This mode also does not allow to use features
-for backend data writing (DB IDO, Perfdata, etc.) as the client does not have
-local objects configured.
-
-Icinga 2 already provides a variety of `CheckCommand` definitions using the Plugin
-Check Commands, but you should also modify the local configuration inside `commands.conf`
-for example.
-
-If you're wondering, why you need to keep the same command configuration on the master and
-remote client: Icinga 2 calculates all required runtime macros used as command arguments on
-the master and sends that information to the client.
-In case you want to limit command arguments or handles values in a different manner, you
-can modify the check command configuration on the remote client only. See [this issue](https://dev.icinga.org/issues/8221#note-3)
-for more details.
-
-### <a id="icinga2-client-configuration-command-bridge-master-config"></a> Master Configuration for Clients as Command Execution Bridge
-
-This step involves little knowledge about the way the Icinga 2 nodes communication and trust
-each other. Each client is configured as `Endpoint` object providing connection information.
-As a matter of trust the client `Endpoint` is a member of its own `Zone` object which gets
-the master zone configured as parent. That way the master knows how to connect to the client
-and where to execute the check commands.
-
-Add an `Endpoint` and `Zone` configuration object for the remote client
-in `/etc/icinga2/zones.conf` and define a trusted master zone as `parent`.
-
- object Endpoint "icinga2-node2.localdomain" {
- host = "192.168.56.102"
- }
-
- object Zone "icinga2-node2.localdomain" {
- parent = "master"
- endpoints = [ "icinga2-node2.localdomain" ]
- }
-
-More details here:
-* [configure endpoints](13-distributed-monitoring-ha.md#configure-cluster-endpoints)
-* [configure zones](13-distributed-monitoring-ha.md#configure-cluster-zones)
-
-
-Once you have configured the required `Endpoint` and `Zone` object definition, you can start
-configuring your host and service objects. The configuration is simple: If the `command_endpoint`
-attribute is set, Icinga 2 calculcates all required runtime macros and sends that over to the
-defined endpoint. The check result is then received asynchronously through the cluster protocol.
-
- object Host "host-remote" {
- import "generic-host"
-
- address = "127.0.0.1"
- address6 = "::1"
-
- vars.os = "Linux"
- }
-
- apply Service "users-remote" {
- import "generic-service"
-
- check_command = "users"
- command_endpoint = "icinga2-node2.localdomain"
-
- vars.users_wgreater = 10
- vars.users_cgreater = 20
-
- /* assign where a remote client pattern is matched */
- assign where match("*-remote", host.name)
- }
-
-
-If there is a failure on execution (for example, the local check command configuration or the plugin
-is missing), the check will return `UNKNOWN` and populate the check output with the error message.
-This will happen in a similar fashion if you forgot to enable the `accept_commands` attribute
-inside the `api` feature.
-
-If you don't want to define the endpoint name inside the service apply rule everytime, you can
-also easily inherit this from a host's custom attribute like shown in the example below.
-
- object Host "host-remote" {
- import "generic-host"
-
- address = "127.0.0.1"
- address6 = "::1"
-
- vars.os = "Linux"
-
- vars.remote_client = "icinga2-node2.localdomain"
-
- /* host specific check arguments */
- vars.users_wgreater = 10
- vars.users_cgreater = 20
- }
-
- apply Service "users-remote" {
- import "generic-service"
-
- check_command = "users"
- command_endpoint = host.vars.remote_client
-
- /* override (remote) command arguments with host settings */
- vars.users_wgreater = host.vars.users_wgreater
- vars.users_cgreater = host.vars.users_cgreater
-
- /* assign where a remote client is set */
- assign where host.vars.remote_client
- }
-
-That way your generated host object is the information provider and the service apply
-rules must only be configured once.
-
-> **Tip**
->
-> [Event commands](3-monitoring-basics.md#event-commands) are executed on the
-> remote command endpoint as well. You do not need
-> an additional transport layer such as SSH or similar.
-
-
-### <a id="icinga2-client-configuration-master-config-sync"></a> Clients with Master Config Sync
-
-This is an advanced configuration mode which requires knowledge about the Icinga 2
-cluster configuration and its object relation (Zones, Endpoints, etc.) and the way you
-will be able to sync the configuration from the master to the remote satellite or client.
-
-Please continue reading in the [distributed monitoring chapter](13-distributed-monitoring-ha.md#distributed-monitoring-high-availability),
-especially the [configuration synchronisation](13-distributed-monitoring-ha.md#cluster-zone-config-sync)
-and [best practices](13-distributed-monitoring-ha.md#zone-config-sync-best-practice).
-
-
-
-
-### <a id="icinga2-client-cli-node"></a> Advanced Node Cli Actions
-
-#### <a id="icinga2-remote-monitoring-master-discovery-blacklist-whitelist"></a> Blacklist/Whitelist for Clients on the Master
-
-It's sometimes necessary to `blacklist` an entire remote client, or specific hosts or services
-provided by this client. While it's reasonable for the local admin to configure for example an
-additional ping check, you're not interested in that on the master sending out notifications
-and presenting the dashboard to your support team.
-
-Blacklisting an entire set might not be sufficient for excluding several objects, be it a
-specific remote client with one ping servie you're interested in. Therefore you can `whitelist`
-clients, hosts, services in a similar manner
-
-Example for blacklisting all `ping*` services, but allowing only `probe` host with `ping4`:
-
- # icinga2 node blacklist add --zone "*" --host "*" --service "ping*"
- # icinga2 node whitelist add --zone "*" --host "probe" --service "ping*"
-
-You can `list` and `remove` existing blacklists:
-
- # icinga2 node blacklist list
- Listing all blacklist entries:
- blacklist filter for Node: '*' Host: '*' Service: 'ping*'.
-
- # icinga2 node whitelist list
- Listing all whitelist entries:
- whitelist filter for Node: '*' Host: 'probe' Service: 'ping*'.
-
-
-> **Note**
->
-> The blacklist feature only prevents future updates from creating and removing objects, but it does not remove already existing objects.
-> The `--zone` and `--host` arguments are required. A zone is always where the remote client is in.
-> If you are unsure about it, set a wildcard (`*`) for them and filter only by host/services.
-
-
-
-#### <a id="icinga2-client-master-discovery-manual"></a> Manually Discover Clients on the Master
-
-Add a to-be-discovered client to the master:
-
- # icinga2 node add my-remote-client
-
-Set the connection details, and the Icinga 2 master will attempt to connect to this node and sync its
-object repository.
-
- # icinga2 node set my-remote-client --host 192.168.33.101 --port 5665
-
-You can control that by calling the `node list` command:
-
- # icinga2 node list
- Node 'my-remote-client' (host: 192.168.33.101, port: 5665, log duration: 1 day, last seen: Sun Nov 2 17:46:29 2014)
-
-#### <a id="icinga2-remote-monitoring-master-discovery-remove"></a> Remove Discovered Clients
-
-If you don't require a connected agent, you can manually remove it and its discovered hosts and services
-using the following CLI command:
-
- # icinga2 node remove my-discovered-agent
-
-> **Note**
->
-> Better use [blacklists and/or whitelists](11-icinga2-client.md#icinga2-remote-monitoring-master-discovery-blacklist-whitelist)
-> to control which clients and hosts/services are integrated into your master configuration repository.
## <a id="icinga2-api-setup"></a> Setting up the API
You can run the CLI command `icinga2 api setup` to enable the
-`api` [feature](8-cli-commands.md#enable-features) and set up
+`api` [feature](11-cli-commands.md#enable-features) and set up
certificates as well as a new API user `root` with an auto-generated password in the
`/etc/icinga2/conf.d/api-users.conf` configuration file:
The URL endpoints are logically separated allowing you to easily
make calls to
-* query, create, modify and delete [config objects](9-icinga2-api.md#icinga2-api-config-objects)
-* perform [actions](9-icinga2-api.md#icinga2-api-actions) (reschedule checks, etc.)
-* subscribe to [event streams](9-icinga2-api.md#icinga2-api-event-streams)
-* [manage configuration packages](9-icinga2-api.md#icinga2-api-config-management)
-* evaluate [script expressions](9-icinga2-api.md#icinga2-api-console)
+* query, create, modify and delete [config objects](12-icinga2-api.md#icinga2-api-config-objects)
+* perform [actions](12-icinga2-api.md#icinga2-api-actions) (reschedule checks, etc.)
+* subscribe to [event streams](12-icinga2-api.md#icinga2-api-event-streams)
+* [manage configuration packages](12-icinga2-api.md#icinga2-api-config-management)
+* evaluate [script expressions](12-icinga2-api.md#icinga2-api-console)
### <a id="icinga2-api-requests"></a> Requests
By default the Icinga 2 API listens on port `5665` which is shared with
the cluster stack. The port can be changed by setting the `bind_port` attribute
-for the [ApiListener](6-object-types.md#objecttype-apilistener)
+for the [ApiListener](9-object-types.md#objecttype-apilistener)
object in the `/etc/icinga2/features-available/api.conf`
configuration file.
* username and password using HTTP basic auth
* X.509 certificate
-In order to configure a new API user you'll need to add a new [ApiUser](6-object-types.md#objecttype-apiuser)
+In order to configure a new API user you'll need to add a new [ApiUser](9-object-types.md#objecttype-apiuser)
configuration object. In this example `root` will be the basic auth username
and the `password` attribute contains the basic auth password.
Alternatively you can use X.509 client certificates by specifying the `client_cn`
the API should trust. The X.509 certificate has to be signed by the CA certificate
-that is configured in the [ApiListener](6-object-types.md#objecttype-apilistener) object.
+that is configured in the [ApiListener](9-object-types.md#objecttype-apilistener) object.
# vim /etc/icinga2/conf.d/api-users.conf
$ curl -u root:icinga --cacert ca.crt 'icinga2.node1.localdomain:5665/v1'
-Read the next chapter on [API permissions](9-icinga2-api.md#icinga2-api-permissions)
+Read the next chapter on [API permissions](12-icinga2-api.md#icinga2-api-permissions)
in order to configure authorization settings for your newly created API user.
### <a id="icinga2-api-permissions"></a> Permissions
permissions = [ "objects/query/Host", "objects/query/Service" ]
You can also further restrict permissions by specifying a filter expression. The
-filter expression has to be a [lambda function](18-language-reference.md#nullary-lambdas)
+filter expression has to be a [lambda function](17-language-reference.md#nullary-lambdas)
which must return a boolean value.
The following example allows the API user to query all hosts and services which have a
}
]
-More information about filters can be found in the [filters](9-icinga2-api.md#icinga2-api-filters) chapter.
+More information about filters can be found in the [filters](12-icinga2-api.md#icinga2-api-filters) chapter.
Available permissions for specific URL endpoints:
For example when querying objects of type `Host` the variable in the filter expression is named
`host`. Additionally related objects such as the host's check command are also made available
(e.g., via the `check_command` variable). The variable names are the exact same as for the `joins`
-query parameter; see [object query joins](9-icinga2-api.md#icinga2-api-config-objects-query-joins)
+query parameter; see [object query joins](12-icinga2-api.md#icinga2-api-config-objects-query-joins)
for details.
The object is also made available via the `obj` variable. This makes it easier to build
Provides methods to manage configuration objects:
-* [creating objects](9-icinga2-api.md#icinga2-api-config-objects-create)
-* [querying objects](9-icinga2-api.md#icinga2-api-config-objects-query)
-* [modifying objects](9-icinga2-api.md#icinga2-api-config-objects-modify)
-* [deleting objects](9-icinga2-api.md#icinga2-api-config-objects-delete)
+* [creating objects](12-icinga2-api.md#icinga2-api-config-objects-create)
+* [querying objects](12-icinga2-api.md#icinga2-api-config-objects-query)
+* [modifying objects](12-icinga2-api.md#icinga2-api-config-objects-modify)
+* [deleting objects](12-icinga2-api.md#icinga2-api-config-objects-delete)
### <a id="icinga2-api-config-objects-cluster-sync"></a> API Objects and Cluster Config Sync
>
> Cluster nodes must accept configuration for creating, modifying
> and deleting objects. Ensure that `accept_config` is set to `true`
-> in the [ApiListener](6-object-types.md#objecttype-apilistener) object
+> in the [ApiListener](9-object-types.md#objecttype-apilistener) object
> on each node.
If you add a new cluster instance, or reconnect an instance which has been offline
$ curl -k -s -u root:icinga 'https://localhost:5665/v1/objects/hosts'
A list of all available configuration types is available in the
-[object types](6-object-types.md#object-types) chapter.
+[object types](9-object-types.md#object-types) chapter.
The following URL parameters are available:
joins | string array | **Optional.** Join related object types and their attributes (`?joins=host` for the entire set, or selectively by `?joins=host.name`).
meta | string array | **Optional.** Enable meta information using `?meta=used_by`. Defaults to disabled.
-In addition to these parameters a [filter](9-icinga2-api.md#icinga2-api-filters) may be provided.
+In addition to these parameters a [filter](12-icinga2-api.md#icinga2-api-filters) may be provided.
Instead of using a filter you can optionally specify the object name in the
URL path when querying a single object. For objects with composite names
name | string | Full object name.
type | string | Object type.
attrs | dictionary | Object attributes (can be filtered using the URL parameter `attrs`).
- joins | dictionary | [Joined object types](9-icinga2-api.md#icinga2-api-config-objects-query-joins) as key, attributes as nested dictionary. Disabled by default.
+ joins | dictionary | [Joined object types](12-icinga2-api.md#icinga2-api-config-objects-query-joins) as key, attributes as nested dictionary. Disabled by default.
meta | dictionary | Contains `used_by` object references. Disabled by default, enable it using `?meta=used_by` as URL parameter.
#### <a id="icinga2-api-config-objects-query-joins"></a> Object Query Joins
]
}
-In case you want to fetch all [comments](6-object-types.md#objecttype-comment)
+In case you want to fetch all [comments](9-object-types.md#objecttype-comment)
for hosts and services, you can use the following query URL (similar example
for downtimes):
Parameters | Type | Description
-----------|--------------|--------------------------
templates | string array | **Optional.** Import existing configuration templates for this object type.
- attrs | dictionary | **Required.** Set specific object attributes for this [object type](6-object-types.md#object-types).
+ attrs | dictionary | **Required.** Set specific object attributes for this [object type](9-object-types.md#object-types).
The object name must be specified as part of the URL path. For objects with composite names (e.g. services)
the full name (e.g. `localhost!http`) must be specified.
Parameters | Type | Description
-----------|------------|---------------------------
- attrs | dictionary | **Required.** Set specific object attributes for this [object type](6-object-types.md#object-types).
+ attrs | dictionary | **Required.** Set specific object attributes for this [object type](9-object-types.md#object-types).
-In addition to these parameters a [filter](9-icinga2-api.md#icinga2-api-filters) should be provided.
+In addition to these parameters a [filter](12-icinga2-api.md#icinga2-api-filters) should be provided.
**Note**: Modified attributes do not trigger a re-evaluation of existing
static [apply rules](3-monitoring-basics.md#using-apply) and [group assignments](3-monitoring-basics.md#group-assign-intro).
-----------|---------|---------------
cascade | boolean | **Optional.** Delete objects depending on the deleted objects (e.g. services on a host).
-In addition to these parameters a [filter](9-icinga2-api.md#icinga2-api-filters) should be provided.
+In addition to these parameters a [filter](12-icinga2-api.md#icinga2-api-filters) should be provided.
Example for deleting the host object `example.localdomain`:
Provides methods to manage configuration templates:
-* [querying templates](9-icinga2-api.md#icinga2-api-config-templates-query)
+* [querying templates](12-icinga2-api.md#icinga2-api-config-templates-query)
### <a id="icinga2-api-config-templates-query"></a> Querying Templates
$ curl -k -s -u root:icinga 'https://localhost:5665/v1/templates/hosts'
A list of all available configuration types is available in the
-[object types](6-object-types.md#object-types) chapter.
+[object types](9-object-types.md#object-types) chapter.
-A [filter](9-icinga2-api.md#icinga2-api-filters) may be provided for this query type. The
+A [filter](12-icinga2-api.md#icinga2-api-filters) may be provided for this query type. The
template object can be accessed in the filter using the `tmpl` variable:
$ curl -u root:root -k 'https://localhost:5661/v1/templates/hosts' -H "Accept: application/json" -X PUT -H "X-HTTP-Method-Override: GET" \
Provides methods to manage global variables:
-* [querying variables](9-icinga2-api.md#icinga2-api-variables-query)
+* [querying variables](12-icinga2-api.md#icinga2-api-variables-query)
### <a id="icinga2-api-variables-query"></a> Querying Variables
$ curl -k -s -u root:icinga 'https://localhost:5665/v1/variables'
-A [filter](9-icinga2-api.md#icinga2-api-filters) may be provided for this query type. The
+A [filter](12-icinga2-api.md#icinga2-api-filters) may be provided for this query type. The
variable information object can be accessed in the filter using the `variable` variable:
$ curl -u root:root -k 'https://localhost:5661/v1/variables' -H "Accept: application/json" -X PUT -H "X-HTTP-Method-Override: GET" \
There are several actions available for Icinga 2 provided by the `/v1/actions`
URL endpoint. You can run actions by sending a `POST` request.
-In case you have been using the [external commands](15-features.md#external-commands)
+In case you have been using the [external commands](14-features.md#external-commands)
in the past, the API actions provide a similar interface with filter
capabilities for some of the more common targets which do not directly change
the configuration.
Actions which affect the Icinga Application itself such as disabling
notification on a program-wide basis must be applied by updating the
-[IcingaApplication object](9-icinga2-api.md#icinga2-api-config-objects)
+[IcingaApplication object](12-icinga2-api.md#icinga2-api-config-objects)
called `app`.
$ curl -k -s -u root:icinga -H 'Accept: application/json' -X POST 'https://localhost:5665/v1/objects/icingaapplications/app' -d '{ "attrs": { "enable_notifications": false } }'
check\_command | string array | **Optional.** The first entry should be the check commands path, then one entry for each command line option followed by an entry for each of its argument.
check\_source | string | **Optional.** Usually the name of the `command_endpoint`
-In addition to these parameters a [filter](9-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
+In addition to these parameters a [filter](12-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
Example:
next\_check | timestamp | **Optional.** The next check will be run at this time. If omitted, the current time is used.
force\_check | boolean | **Optional.** Defaults to `false`. If enabled, the checks are executed regardless of time period restrictions and checks being disabled per object or on a global basis.
-In addition to these parameters a [filter](9-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
+In addition to these parameters a [filter](12-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
The example reschedules all services with the name "ping6" to immediately perform a check
(`next_check` default), ignoring any time periods or whether active checks are
comment | string | **Required.** Comment text, may be empty.
force | boolean | **Optional.** Default: false. If true, the notification is sent regardless of downtimes or whether notifications are enabled or not.
-In addition to these parameters a [filter](9-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
+In addition to these parameters a [filter](12-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
Example for a custom host notification announcing a global maintenance to
host owners:
----------|-----------|--------------
timestamp | timestamp | **Required.** Delay notifications until this timestamp.
-In addition to these parameters a [filter](9-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
+In addition to these parameters a [filter](12-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
Example:
sticky | boolean | **Optional.** If `true`, the default, the acknowledgement will remain until the service or host fully recovers.
notify | boolean | **Optional.** If `true`, a notification will be sent out to contacts to indicate this problem has been acknowledged. The default is false.
-In addition to these parameters a [filter](9-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
+In addition to these parameters a [filter](12-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
The following example acknowledges all services which are in a hard critical state and sends out
a notification for them:
Send a `POST` request to the URL endpoint `/v1/actions/remove-acknowledgement`.
-A [filter](9-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
+A [filter](12-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
The example removes all service acknowledgements:
author | string | **Required.** Name of the author, may be empty.
comment | string | **Required.** Comment text, may be empty.
-In addition to these parameters a [filter](9-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
+In addition to these parameters a [filter](12-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
The following example adds a comment for all `ping4` services:
Remove the comment using its `name` attribute , returns `OK` if the
comment did not exist.
**Note**: This is **not** the legacy ID but the comment name returned by
-Icinga 2 when [adding a comment](9-icinga2-api.md#icinga2-api-actions-add-comment).
+Icinga 2 when [adding a comment](12-icinga2-api.md#icinga2-api-actions-add-comment).
Send a `POST` request to the URL endpoint `/v1/actions/remove-comment`.
-A [filter](9-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host`, `Service` and `Comment`.
+A [filter](12-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host`, `Service` and `Comment`.
Example for a simple filter using the `comment` URL parameter:
start\_time | timestamp | **Required.** Timestamp marking the beginning of the downtime.
end\_time | timestamp | **Required.** Timestamp marking the end of the downtime.
duration | integer | **Required.** Duration of the downtime in seconds if `fixed` is set to false.
- fixed | boolean | **Optional.** Defaults to `true`. If true, the downtime is `fixed` otherwise `flexible`. See [downtimes](5-advanced-topics.md#downtimes) for more information.
- trigger\_name | string | **Optional.** Sets the trigger for a triggered downtime. See [downtimes](5-advanced-topics.md#downtimes) for more information on triggered downtimes.
+ fixed | boolean | **Optional.** Defaults to `true`. If true, the downtime is `fixed` otherwise `flexible`. See [downtimes](8-advanced-topics.md#downtimes) for more information.
+ trigger\_name | string | **Optional.** Sets the trigger for a triggered downtime. See [downtimes](8-advanced-topics.md#downtimes) for more information on triggered downtimes.
-In addition to these parameters a [filter](9-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
+In addition to these parameters a [filter](12-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host` and `Service`.
Example:
Remove the downtime using its `name` attribute , returns `OK` if the
downtime did not exist.
**Note**: This is **not** the legacy ID but the downtime name returned by
-Icinga 2 when [scheduling a downtime](9-icinga2-api.md#icinga2-api-actions-schedule-downtime).
+Icinga 2 when [scheduling a downtime](12-icinga2-api.md#icinga2-api-actions-schedule-downtime).
Send a `POST` request to the URL endpoint `/v1/actions/remove-downtime`.
-A [filter](9-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host`, `Service` and `Downtime`.
+A [filter](12-icinga2-api.md#icinga2-api-filters) must be provided. The valid types for this action are `Host`, `Service` and `Downtime`.
Example for a simple filter using the `downtime` URL parameter:
}
Example for removing a downtime from a host but not the services filtered by the author name. This example uses
-filter variables explained in the [advanced filters](9-icinga2-api.md#icinga2-api-advanced-filters) chapter.
+filter variables explained in the [advanced filters](12-icinga2-api.md#icinga2-api-advanced-filters) chapter.
$ curl -k -s -u root:icinga -H 'Accept: application/json' -X POST 'https://localhost:5665/v1/actions/remove-downtime' \
-d $'{
-----------|--------------|-------------
types | string array | **Required.** Event type(s). Multiple types as URL parameters are supported.
queue | string | **Required.** Unique queue name. Multiple HTTP clients can use the same queue as long as they use the same event types and filter.
- filter | string | **Optional.** Filter for specific event attributes using [filter expressions](9-icinga2-api.md#icinga2-api-filters).
+ filter | string | **Optional.** Filter for specific event attributes using [filter expressions](12-icinga2-api.md#icinga2-api-filters).
### <a id="icinga2-api-event-streams-types"></a> Event Stream Types
DowntimeRemoved | Downtime removed for hosts and services.
DowntimeTriggered | Downtime triggered for hosts and services.
-Note: Each type requires [API permissions](9-icinga2-api.md#icinga2-api-permissions)
+Note: Each type requires [API permissions](12-icinga2-api.md#icinga2-api-permissions)
being set.
Example for all downtime events:
Directory | Description
------------|------------------------------------
conf.d | Local configuration directory.
- zones.d | Configuration directory for cluster zones, each zone must be put into its own zone directory underneath. Supports the [cluster config sync](13-distributed-monitoring-ha.md#cluster-zone-config-sync).
+ zones.d | Configuration directory for cluster zones, each zone must be put into its own zone directory underneath. Supports the [cluster config sync](6-distributed-monitoring.md#distributed-monitoring-top-down-config-sync).
Example for a local configuration in the `conf.d` directory:
> **Note**
>
-> Old stages are not purged automatically. You can [remove stages](9-icinga2-api.md#) that are no longer in use.
+> Old stages are not purged automatically. You can [remove stages](12-icinga2-api.md#icinga2-api-config-management-delete-config-stage) that are no longer in use.
Icinga 2 will create the following files in the configuration package
stage after configuration validation:
File | Description
------------|--------------
- status | Contains the [configuration validation](8-cli-commands.md#config-validation) exit code (everything else than 0 indicates an error).
- startup.log | Contains the [configuration validation](8-cli-commands.md#config-validation) output.
+ status | Contains the [configuration validation](11-cli-commands.md#config-validation) exit code (everything else than 0 indicates an error).
+ startup.log | Contains the [configuration validation](11-cli-commands.md#config-validation) output.
-You can [fetch these files](9-icinga2-api.md#icinga2-api-config-management-fetch-config-package-stage-files)
+You can [fetch these files](12-icinga2-api.md#icinga2-api-config-management-fetch-config-package-stage-files)
in order to verify that the new configuration was deployed successfully.
object Host "cmdb-host" { chec_command = "dummy" }
-You can fetch a [list of existing files](9-icinga2-api.md#icinga2-api-config-management-list-config-package-stage-files)
+You can fetch a [list of existing files](12-icinga2-api.md#icinga2-api-config-management-list-config-package-stage-files)
in a configuration stage and then specifically request their content.
### <a id="icinga2-api-config-management-config-package-stage-errors"></a> Configuration Package Stage Errors
-Now that we don't have an active stage for `example-cmdb` yet seen [here](9-icinga2-api.md#icinga2-api-config-management-list-config-packages),
+Now that we don't have an active stage for `example-cmdb` yet seen [here](12-icinga2-api.md#icinga2-api-config-management-list-config-packages),
there must have been an error.
In order to check for validation errors you can fetch the `startup.log` file
critical/config: 1 error
-The output is similar to the manual [configuration validation](8-cli-commands.md#config-validation).
+The output is similar to the manual [configuration validation](11-cli-commands.md#config-validation).
> **Note**
>
command | string | **Required.** Command expression for execution or auto-completion.
sandboxed | number | **Optional.** Whether runtime changes are allowed or forbidden. Defaults to disabled.
-The [API permission](9-icinga2-api.md#icinga2-api-permissions) `console` is required for executing
+The [API permission](12-icinga2-api.md#icinga2-api-permissions) `console` is required for executing
expressions.
If you specify a session identifier, the same script context can be reused for multiple requests. This allows you to, for example, set a local variable in a request and use that local variable in another request. Sessions automatically expire after a set period of inactivity (currently 30 minutes).
}
Example for fetching auto-completion suggestions for the `Host.` type. This works in a
-similar fashion when pressing TAB inside the [console CLI command](8-cli-commands.md#cli-command-console):
+similar fashion when pressing TAB inside the [console CLI command](11-cli-commands.md#cli-command-console):
$ curl -k -s -u root:icinga -H 'Accept: application/json' -X POST 'https://localhost:5665/v1/console/auto-complete-script?command=Host.&sandboxed=0&session=bb75fd7c-c686-407d-9688-582c04227756' | python -m json.tool
{
There are a couple of existing clients which can be used with the Icinga 2 API:
* [curl](http://curl.haxx.se) or any other HTTP client really
-* [Icinga 2 console (CLI command)](9-icinga2-api.md#icinga2-api-clients-cli-console)
-* [Icinga Studio](9-icinga2-api.md#icinga2-api-clients-icinga-studio)
+* [Icinga 2 console (CLI command)](12-icinga2-api.md#icinga2-api-clients-cli-console)
+* [Icinga Studio](12-icinga2-api.md#icinga2-api-clients-icinga-studio)
* [Icinga Web 2 Director](https://dev.icinga.org/projects/icingaweb2-modules)
Demo cases:
* [Dashing](https://github.com/Icinga/dashing-icinga2)
* [API examples](https://github.com/Icinga/icinga2-api-examples)
-Additional [programmatic examples](9-icinga2-api.md#icinga2-api-clients-programmatic-examples)
+Additional [programmatic examples](12-icinga2-api.md#icinga2-api-clients-programmatic-examples)
will help you getting started using the Icinga 2 API in your environment.
### <a id="icinga2-api-clients-icinga-studio"></a> Icinga Studio
### <a id="icinga2-api-clients-cli-console"></a> Icinga 2 Console
-By default the [console CLI command](8-cli-commands.md#cli-command-console) evaluates expressions in a local interpreter, i.e. independently from your Icinga 2 daemon. Using the `--connect` parameter you can use the Icinga 2 console to evaluate expressions via the API.
+By default the [console CLI command](11-cli-commands.md#cli-command-console) evaluates expressions in a local interpreter, i.e. independently from your Icinga 2 daemon. Using the `--connect` parameter you can use the Icinga 2 console to evaluate expressions via the API.
### <a id="icinga2-api-clients-programmatic-examples"></a> API Clients Programmatic Examples
-# <a id="addons-plugins"></a> Icinga 2 Addons and Plugins
+# <a id="addons"></a> Icinga 2 Addons
## <a id="addons-graphing"></a> Graphing
If you're planning to use it, configure it to use the
[bulk mode with npcd and npcdmod](http://docs.pnp4nagios.org/pnp-0.6/modes#bulk_mode_with_npcd_and_npcdmod)
-in combination with Icinga 2's [PerfdataWriter](15-features.md#performance-data). NPCD collects the performance
+in combination with Icinga 2's [PerfdataWriter](14-features.md#performance-data). NPCD collects the performance
data files which Icinga 2 generates.
Enable performance data writer in icinga 2
There's also an Icinga Web 2 module for direct PNP graph integration
available at [Icinga Exchange](https://exchange.icinga.org/icinga/PNP).
-More information on [action_url as attribute](14-addons-plugins.md#addons-graphing-pnp-action-url)
-and [graph template names](14-addons-plugins.md#addons-graphing-pnp-custom-templates).
+More information on [action_url as attribute](13-addons.md#addons-graphing-pnp-action-url)
+and [graph template names](13-addons.md#addons-graphing-pnp-custom-templates).
### <a id="addons-graphing-graphite"></a> Graphite
* whisper -- a simple database library for storing time-series data (similar in design to RRD)
* graphite webapp -- a Django webapp that renders graphs on-demand using Cairo
-Use the [GraphiteWriter](15-features.md#graphite-carbon-cache-writer) feature
+Use the [GraphiteWriter](14-features.md#graphite-carbon-cache-writer) feature
for sending real-time metrics from Icinga 2 to Graphite.
# icinga2 feature enable graphite
[InfluxDB](https://influxdb.com) is a time series, metrics, and analytics database.
It’s written in Go and has no external dependencies.
-Use the [InfluxdbWriter](15-features.md#influxdb-writer) feature
+Use the [InfluxdbWriter](14-features.md#influxdb-writer) feature
for sending real-time metrics from Icinga 2 to InfluxDB.
# icinga2 feature enable influxdb
### <a id="addons-visualization-reporting"></a> Icinga Reporting
-By enabling the [DB IDO](15-features.md#db-ido) feature you can use the
+By enabling the [DB IDO](14-features.md#db-ido) feature you can use the
[Icinga Reporting package](https://wiki.icinga.org/display/howtos/Setting+up+Icinga+with+Reporting).
### <a id="addons-visualization-nagvis"></a> NagVis
-By using either [Livestatus](15-features.md#setting-up-livestatus) or
-[DB IDO](15-features.md#db-ido) as a backend you can create your own network maps
+By using either [Livestatus](14-features.md#setting-up-livestatus) or
+[DB IDO](14-features.md#db-ido) as a backend you can create your own network maps
based on your monitoring configuration and status data using [NagVis](http://www.nagvis.org).
The configuration in nagvis.ini.php should look like this for Livestatus for example:
### <a id="addons-visualization-thruk"></a> Thruk
[Thruk](http://www.thruk.org) is an alternative web interface which can be used with Icinga 2
-and the [Livestatus](15-features.md#setting-up-livestatus) feature.
+and the [Livestatus](14-features.md#setting-up-livestatus) feature.
## <a id="log-monitoring"></a> Log Monitoring
* [Puppet Module](https://github.com/Icinga/puppet-icinga2)
* [Chef Cookbook](https://github.com/Icinga/chef-icinga2)
-## <a id="plugins"></a> Plugins
-
-For some services you may need additional 'check plugins' which are not provided
-by the official Monitoring Plugins project.
-
-All existing Nagios or Icinga 1.x plugins work with Icinga 2. Here's a
-list of popular community sites which host check plugins:
-
-* [Icinga Exchange](https://exchange.icinga.org)
-* [Icinga Wiki](https://wiki.icinga.org)
-
-The recommended way of setting up these plugins is to copy them to a common directory
-and create a new global constant, e.g. `CustomPluginDir` in your [constants.conf](4-configuring-icinga-2.md#constants-conf)
-configuration file:
-
- # cp check_snmp_int.pl /opt/monitoring/plugins
- # chmod +x /opt/plugins/check_snmp_int.pl
-
- # cat /etc/icinga2/constants.conf
- /**
- * This file defines global constants which can be used in
- * the other configuration files. At a minimum the
- * PluginDir constant should be defined.
- */
-
- const PluginDir = "/usr/lib/nagios/plugins"
- const CustomPluginDir = "/opt/monitoring/plugins"
-
-Prior to using the check plugin with Icinga 2 you should ensure that it is working properly
-by trying to run it on the console using whichever user Icinga 2 is running as:
-
- # su - icinga -s /bin/bash
- $ /opt/monitoring/plugins/check_snmp_int.pl --help
-
-Additional libraries may be required for some plugins. Please consult the plugin
-documentation and/or plugin provided README for installation instructions.
-Sometimes plugins contain hard-coded paths to other components. Instead of changing
-the plugin it might be easier to create logical links which is (more) update-safe.
-
-Each plugin requires a [CheckCommand](6-object-types.md#objecttype-checkcommand) object in your
-configuration which can be used in the [Service](6-object-types.md#objecttype-service) or
-[Host](6-object-types.md#objecttype-host) object definition.
-
-There are the following conventions to follow when adding a new command object definition:
-
-* Always import the `plugin-check-command` template
-* Use [command-arguments](#) whenever possible. The `command` attribute must be an array
-in `[ ... ]` then for shell escaping.
-* Define a unique `prefix` for the command's specific command arguments. That way you can safely
-set them on host/service level and you'll always know which command they control.
-* Use command argument default values, e.g. for thresholds
-* Use [advanced conditions](6-object-types.md#objecttype-checkcommand) like `set_if` definitions.
-
-Example for a custom `my-snmp-int` check command:
-
- object CheckCommand "my-snmp-int" {
- import "plugin-check-command"
-
- command = [ CustomPluginDir + "/check_snmp_int.pl" ]
-
- arguments = {
- "-H" = "$snmp_address$"
- "-C" = "$snmp_community$"
- "-p" = "$snmp_port$"
- "-2" = {
- set_if = "$snmp_v2$"
- }
- "-n" = "$snmp_interface$"
- "-f" = {
- set_if = "$snmp_perf$"
- }
- "-w" = "$snmp_warn$"
- "-c" = "$snmp_crit$"
- }
-
- vars.snmp_v2 = true
- vars.snmp_perf = true
- vars.snmp_warn = "300,400"
- vars.snmp_crit = "0,600"
- }
-
-Icinga 2 has built-in check command definitions for the [Manubulon Plugin Checks](7-icinga-template-library.md#snmp-manubulon-plugin-check-commands).
-
-For further information on your monitoring configuration read the
-[Monitoring Basics](3-monitoring-basics.md#monitoring-basics) chapter.
-
-You can find additional plugins at the [Icinga Exchange](https://exchange.icinga.org)
-
-More details on the plugins can also be found on the Icinga Wiki at https://wiki.icinga.org
-
-> **Tip**
->
-> Create the best `CheckCommand` definition there is and send it upstream. More
-> information can be found in [Contribute Icinga 2 ITL Plugin Check Command Definitions](https://wiki.icinga.org/display/community/Contribute+Icinga+2+ITL+Plugin+Check+Command+Definitions)
-> on the Icinga Wiki. Thank you in advance!
-
-## <a id="plugin-api"></a> Plugin API
-
-Currently Icinga 2 supports the native plugin API specification from the `Monitoring Plugins`
-project.
-
-The `Monitoring Plugin API` is defined in the [Monitoring Plugins Development Guidelines](https://www.monitoring-plugins.org/doc/guidelines.html).
-
-There are no output length restrictions using Icinga 2. This is different to the
-[Icinga 1.x plugin api definition](http://docs.icinga.org/latest/en/pluginapi.html#outputlengthrestrictions).
-
-
## <a id="addon-integration-hints"></a> More Addon Integration Hints
### <a id="addons-graphing-pnp-action-url"></a> PNP Action Url
fix this:
* Create a symlink for example from the `templates.dist/check_ping.php` template to the actual check name in Icinga 2 (`templates/ping4.php`)
-* Pass the check command name inside the [format template configuration](15-features.md#writing-performance-data-files)
+* Pass the check command name inside the [format template configuration](14-features.md#writing-performance-data-files)
The latter becomes difficult with agent based checks like NRPE or SSH where the first command argument acts as
graph template identifier. There is the possibility to define the pnp template name as custom attribute
+++ /dev/null
-# <a id="distributed-monitoring-high-availability"></a> Distributed Monitoring and High Availability
-
-Building distributed environments with high availability included is fairly easy with Icinga 2.
-The cluster feature is built-in and allows you to build many scenarios based on your requirements:
-
-* [High Availability](13-distributed-monitoring-ha.md#cluster-scenarios-high-availability). All instances in the `Zone` run as Active/Active cluster.
-* [Distributed Zones](13-distributed-monitoring-ha.md#cluster-scenarios-distributed-zones). A master zone and one or more satellites in their zones.
-* [Load Distribution](13-distributed-monitoring-ha.md#cluster-scenarios-load-distribution). A configuration master and multiple checker satellites.
-
-You can combine these scenarios into a global setup fitting your requirements.
-
-Each instance got their own event scheduler, and does not depend on a centralized master
-coordinating and distributing the events. In case of a cluster failure, all nodes
-continue to run independently. Be alarmed when your cluster fails and a Split-Brain-scenario
-is in effect -- all alive instances continue to do their job, and history will begin to differ.
-
-
-## <a id="cluster-requirements"></a> Cluster Requirements
-
-Before you start deploying, keep the following things in mind:
-
-Your [SSL CA and certificates](13-distributed-monitoring-ha.md#manual-certificate-generation) are mandatory for secure communication.
-
-Communication between zones requires one of these connection directions:
-
-* The parent zone nodes are able to connect to the child zone nodes (`parent => child`).
-* The child zone nodes are able to connect to the parent zone nodes (`parent <= child`).
-* Both connnection directions work.
-
-Update firewall rules and ACLs.
-
-* Icinga 2 master, satellite and client instances communicate using the default tcp port `5665`.
-
-Get pen and paper or a drawing board and design your nodes and zones!
-
-* Keep the [naming convention](13-distributed-monitoring-ha.md#cluster-naming-convention) for nodes in mind.
-* All nodes (endpoints) in a cluster zone provide high availability functionality and trust each other.
-* Cluster zones can be built in a Top-Down-design where the child trusts the parent.
-
-Decide whether to use the built-in [configuration syncronization](13-distributed-monitoring-ha.md#cluster-zone-config-sync) or use an external tool (Puppet, Ansible, Chef, Salt, etc.) to manage the configuration deployment.
-
-
-> **Tip**
->
-> If you're looking for troubleshooting cluster problems, check the general
-> [troubleshooting](16-troubleshooting.md#troubleshooting-cluster) section.
-
-## <a id="manual-certificate-generation"></a> Manual SSL Certificate Generation
-
-Icinga 2 provides [CLI commands](8-cli-commands.md#cli-command-pki) assisting with CA
-and node certificate creation for your Icinga 2 distributed setup.
-
-You can also use the master and client setup wizards to install the cluster nodes
-using CSR-Autosigning.
-
-The manual steps are helpful if you want to use your own and/or existing CA (for example
-Puppet CA).
-
-You're free to use your own method to generated a valid ca and signed client
-certificates.
-
-The first step is the creation of the certificate authority (CA) by running the
-following command:
-
- # icinga2 pki new-ca
-
-Now create a certificate and key file for each node running the following command
-(replace `icinga2a` with the required hostname):
-
- # icinga2 pki new-cert --cn icinga2a --key icinga2a.key --csr icinga2a.csr
- # icinga2 pki sign-csr --csr icinga2a.csr --cert icinga2a.crt
-
-Repeat the step for all nodes in your cluster scenario.
-
-Save the CA key in a secure location in case you want to set up certificates for
-additional nodes at a later time.
-
-Navigate to the location of your newly generated certificate files, and manually
-copy/transfer them to `/etc/icinga2/pki` in your Icinga 2 configuration folder.
-
-> **Note**
->
-> The certificate files must be readable by the user Icinga 2 is running as. Also,
-> the private key file must not be world-readable.
-
-Each node requires the following files in `/etc/icinga2/pki` (replace `fqdn-nodename` with
-the host's FQDN):
-
-* ca.crt
-* <fqdn-nodename>.crt
-* <fqdn-nodename>.key
-
-If you're planning to use your existing CA and certificates, please note that you *must not*
-use wildcard certificates. The common name (CN) is mandatory for the cluster communication and
-therefore must be unique for each connecting instance.
-
-## <a id="cluster-naming-convention"></a> Cluster Naming Convention
-
-The SSL certificate common name (CN) will be used by the [ApiListener](6-object-types.md#objecttype-apilistener)
-object to determine the local authority. This name must match the local [Endpoint](6-object-types.md#objecttype-endpoint)
-object name.
-
-Certificate generation for host with the FQDN `icinga2a`:
-
- # icinga2 pki new-cert --cn icinga2a --key icinga2a.key --csr icinga2a.csr
- # icinga2 pki sign-csr --csr icinga2a.csr --cert icinga2a.crt
-
-Add a new `Endpoint` object named `icinga2a`:
-
- # vim zones.conf
-
- object Endpoint "icinga2a" {
- host = "icinga2a.icinga.org"
- }
-
-The [Endpoint](6-object-types.md#objecttype-endpoint) name is further referenced as `endpoints` attribute on the
-[Zone](6-object-types.md#objecttype-zone) object.
-
- object Endpoint "icinga2b" {
- host = "icinga2b.icinga.org"
- }
-
- object Zone "config-ha-master" {
- endpoints = [ "icinga2a", "icinga2b" ]
- }
-
-Specifying the local node name using the [NodeName](13-distributed-monitoring-ha.md#configure-nodename) variable requires
-the same name as used for the endpoint name and common name above. If not set, the FQDN is used.
-
- const NodeName = "icinga2a"
-
-If you're using the host's FQDN everywhere, you're on the safe side. The setup wizards
-will do the very same.
-
-## <a id="cluster-configuration"></a> Cluster Configuration
-
-The following section describe which configuration must be updated/created
-in order to get your cluster running with basic functionality.
-
-* [configure the node name](13-distributed-monitoring-ha.md#configure-nodename)
-* [configure the ApiListener object](13-distributed-monitoring-ha.md#configure-apilistener-object)
-* [configure cluster endpoints](13-distributed-monitoring-ha.md#configure-cluster-endpoints)
-* [configure cluster zones](13-distributed-monitoring-ha.md#configure-cluster-zones)
-
-Once you're finished with the basic setup the following section will
-describe how to use [zone configuration synchronisation](13-distributed-monitoring-ha.md#cluster-zone-config-sync)
-and configure [cluster scenarios](13-distributed-monitoring-ha.md#cluster-scenarios).
-
-### <a id="configure-nodename"></a> Configure the Icinga Node Name
-
-Instead of using the default FQDN as node name you can optionally set
-that value using the [NodeName](18-language-reference.md#constants) constant.
-
-> ** Note **
->
-> Skip this step if your FQDN already matches the default `NodeName` set
-> in `/etc/icinga2/constants.conf`.
-
-This setting must be unique for each node, and must also match
-the name of the local [Endpoint](6-object-types.md#objecttype-endpoint) object and the
-SSL certificate common name as described in the
-[cluster naming convention](13-distributed-monitoring-ha.md#cluster-naming-convention).
-
- vim /etc/icinga2/constants.conf
-
- /* Our local instance name. By default this is the server's hostname as returned by `hostname --fqdn`.
- * This should be the common name from the API certificate.
- */
- const NodeName = "icinga2a"
-
-
-Read further about additional [naming conventions](13-distributed-monitoring-ha.md#cluster-naming-convention).
-
-Not specifying the node name will make Icinga 2 using the FQDN. Make sure that all
-configured endpoint names and common names are in sync.
-
-### <a id="configure-apilistener-object"></a> Configure the ApiListener Object
-
-The [ApiListener](6-object-types.md#objecttype-apilistener) object needs to be configured on
-every node in the cluster with the following settings:
-
-A sample config looks like:
-
- object ApiListener "api" {
- cert_path = SysconfDir + "/icinga2/pki/" + NodeName + ".crt"
- key_path = SysconfDir + "/icinga2/pki/" + NodeName + ".key"
- ca_path = SysconfDir + "/icinga2/pki/ca.crt"
- accept_config = true
- accept_commands = true
- }
-
-You can simply enable the `api` feature using
-
- # icinga2 feature enable api
-
-Edit `/etc/icinga2/features-enabled/api.conf` if you require the configuration
-synchronisation enabled for this node. Set the `accept_config` attribute to `true`.
-
-If you want to use this node as [remote client for command execution](11-icinga2-client.md#icinga2-client-configuration-command-bridge),
-set the `accept_commands` attribute to `true`.
-
-> **Note**
->
-> The certificate files must be readable by the user Icinga 2 is running as. Also,
-> the private key file must not be world-readable.
-
-### <a id="configure-cluster-endpoints"></a> Configure Cluster Endpoints
-
-`Endpoint` objects specify the `host` and `port` settings for the cluster node
-connection information.
-This configuration can be the same on all nodes in the cluster only containing
-connection information.
-
-A sample configuration looks like:
-
- /**
- * Configure config master endpoint
- */
-
- object Endpoint "icinga2a" {
- host = "icinga2a.icinga.org"
- }
-
-If this endpoint object is reachable on a different port, you must configure the
-`ApiListener` on the local `Endpoint` object accordingly too.
-
-If you don't want the local instance to connect to the remote instance, remove the
-`host` attribute locally. Keep in mind that the configuration is now different amongst
-all instances and point-of-view dependant.
-
-### <a id="configure-cluster-zones"></a> Configure Cluster Zones
-
-`Zone` objects specify the endpoints located in a zone. That way your distributed setup can be
-seen as zones connected together instead of multiple instances in that specific zone.
-
-Zones can be used for [high availability](13-distributed-monitoring-ha.md#cluster-scenarios-high-availability),
-[distributed setups](13-distributed-monitoring-ha.md#cluster-scenarios-distributed-zones) and
-[load distribution](13-distributed-monitoring-ha.md#cluster-scenarios-load-distribution).
-Furthermore zones are used for the [Icinga 2 remote client](11-icinga2-client.md#icinga2-client).
-
-Each Icinga 2 `Endpoint` must be put into its respective `Zone`. In this example, you will
-define the zone `config-ha-master` where the `icinga2a` and `icinga2b` endpoints
-are located. The `check-satellite` zone consists of `icinga2c` only, but more nodes could
-be added.
-
-The `config-ha-master` zone acts as High-Availability setup -- the Icinga 2 instances elect
-one instance running a check, notification or feature (DB IDO), for example `icinga2a`. In case of
-failure of the `icinga2a` instance, `icinga2b` will take over automatically.
-
- object Zone "config-ha-master" {
- endpoints = [ "icinga2a", "icinga2b" ]
- }
-
-The `check-satellite` zone is a separated location and only sends back their checkresults to
-the defined parent zone `config-ha-master`.
-
- object Zone "check-satellite" {
- endpoints = [ "icinga2c" ]
- parent = "config-ha-master"
- }
-
-
-## <a id="cluster-zone-config-sync"></a> Zone Configuration Synchronisation
-
-In case you are using the Icinga 2 API for creating, modifying and deleting objects
-at runtime, please continue over [here](9-icinga2-api.md#icinga2-api-config-objects-cluster-sync).
-
-By default all objects for specific zones should be organized in
-
- /etc/icinga2/zones.d/<zonename>
-
-on the configuration master.
-
-Your child zones and endpoint members **must not** have their config copied to `zones.d`.
-The built-in configuration synchronisation takes care of that if your nodes accept
-configuration from the parent zone. You can define that in the
-[ApiListener](13-distributed-monitoring-ha.md#configure-apilistener-object) object by configuring the `accept_config`
-attribute accordingly.
-
-You should remove the sample config included in `conf.d` by commenting the `recursive_include`
-statement in [icinga2.conf](4-configuring-icinga-2.md#icinga2-conf):
-
- //include_recursive "conf.d"
-
-This applies to any other non-used configuration directories as well (e.g. `repository.d`
-if not used).
-
-Better use a dedicated directory name for local configuration like `local` or similar, and
-include that one if your nodes require local configuration not being synced to other nodes. That's
-useful for local [health checks](13-distributed-monitoring-ha.md#cluster-health-check) for example.
-
-> **Note**
->
-> In a [high availability](13-distributed-monitoring-ha.md#cluster-scenarios-high-availability)
-> setup only one assigned node can act as configuration master. All other zone
-> member nodes **must not** have the `/etc/icinga2/zones.d` directory populated.
-
-
-These zone packages are then distributed to all nodes in the same zone, and
-to their respective target zone instances.
-
-Each configured zone must exist with the same directory name. The parent zone
-syncs the configuration to the child zones if allowed using the `accept_config`
-attribute of the [ApiListener](13-distributed-monitoring-ha.md#configure-apilistener-object) object.
-
-Config on node `icinga2a`:
-
- object Zone "master" {
- endpoints = [ "icinga2a" ]
- }
-
- object Zone "checker" {
- endpoints = [ "icinga2b" ]
- parent = "master"
- }
-
- /etc/icinga2/zones.d
- master
- health.conf
- checker
- health.conf
- demo.conf
-
-Config on node `icinga2b`:
-
- object Zone "master" {
- endpoints = [ "icinga2a" ]
- }
-
- object Zone "checker" {
- endpoints = [ "icinga2b" ]
- parent = "master"
- }
-
- /etc/icinga2/zones.d
- EMPTY_IF_CONFIG_SYNC_ENABLED
-
-If the local configuration is newer than the received update, Icinga 2 will skip the synchronisation
-process.
-
-> **Note**
->
-> `zones.d` must not be included in [icinga2.conf](4-configuring-icinga-2.md#icinga2-conf). Icinga 2 automatically
-> determines the required include directory. This can be overridden using the
-> [global constant](18-language-reference.md#constants) `ZonesDir`.
-
-### <a id="zone-global-config-templates"></a> Global Configuration Zone for Templates
-
-If your zone configuration setup shares the same templates, groups, commands, timeperiods, etc.,
-you would have to duplicate quite a lot of configuration objects making the merged configuration
-on your configuration master unique.
-
-> ** Note **
->
-> Only put templates, groups, etc. into this zone. DO NOT add checkable objects such as
-> hosts or services here. If they are checked by all instances globally, this will lead
-> into duplicated check results and unclear state history. Not easy to troubleshoot too -
-> you have been warned.
-
-That is not necessary by defining a global zone shipping all those templates. By setting
-`global = true` you ensure that this zone serving common configuration templates will be
-synchronized to all involved nodes (only if they accept configuration though).
-
-Config on configuration master:
-
- /etc/icinga2/zones.d
- global-templates/
- templates.conf
- groups.conf
- master
- health.conf
- checker
- health.conf
- demo.conf
-
-In this example, the global zone is called `global-templates` and must be defined in
-your zone configuration visible to all nodes.
-
- object Zone "global-templates" {
- global = true
- }
-
-If the remote node does not have this zone configured, it will ignore the configuration
-update if it accepts synchronized configuration.
-
-If you do not require any global configuration, skip this setting.
-
-### <a id="zone-config-sync-permissions"></a> Zone Configuration Synchronisation Permissions
-
-Each [ApiListener](6-object-types.md#objecttype-apilistener) object must have the `accept_config` attribute
-set to `true` to receive configuration from the parent `Zone` members. Default value is `false`.
-
- object ApiListener "api" {
- cert_path = SysconfDir + "/icinga2/pki/" + NodeName + ".crt"
- key_path = SysconfDir + "/icinga2/pki/" + NodeName + ".key"
- ca_path = SysconfDir + "/icinga2/pki/ca.crt"
- accept_config = true
- }
-
-If `accept_config` is set to `false`, this instance won't accept configuration from remote
-master instances anymore.
-
-> ** Tip **
->
-> Look into the [troubleshooting guides](16-troubleshooting.md#troubleshooting-cluster-config-sync) for debugging
-> problems with the configuration synchronisation.
-
-
-### <a id="zone-config-sync-best-practice"></a> Zone Configuration Synchronisation Best Practice
-
-The configuration synchronisation works with multiple hierarchies. The following example
-illustrate a quite common setup where the master is reponsible for configuration deployment:
-
-* [High-Availability master zone](13-distributed-monitoring-ha.md#distributed-monitoring-high-availability)
-* [Distributed satellites](13-distributed-monitoring-ha.md#cluster-scenarios-distributed-zones)
-* [Remote clients](11-icinga2-client.md#icinga2-client-scenarios) connected to the satellite
-
-While you could use the clients with local configuration and service discovery on the satellite/master
-**bottom up**, the configuration sync could be more reasonable working **top-down** in a cascaded scenario.
-
-Take pen and paper and draw your network scenario including the involved zone and endpoint names.
-Once you've added them to your zones.conf as connection and permission configuration, start over with
-the actual configuration organization:
-
-* Ensure that `command` object definitions are globally available. That way you can use the
-`command_endpoint` configuration more easily on clients as [command execution bridge](11-icinga2-client.md#icinga2-client-configuration-command-bridge)
-* Generic `Templates`, `timeperiods`, `downtimes` should be synchronized in a global zone as well.
-* [Apply rules](3-monitoring-basics.md#using-apply) can be synchronized globally. Keep in mind that they are evaluated on each instance,
-and might require additional filters (e.g. `match("icinga2*", NodeName) or similar based on the zone information.
-* Host configuration must be put into the specific zone directory.
-* Duplicated host and service objects (also generated by apply rules) will generate a configuration error.
-* Consider using custom constants in your host/service configuration. Each instance may set their local value, e.g. for `PluginDir`.
-
-This example specifies the following hierarchy over three levels:
-
-* `ha-master` zone with two child zones `dmz1-checker` and `dmz2-checker`
-* `dmz1-checker` has two client child zones `dmz1-client1` and `dmz1-client2`
-* `dmz2-checker` has one client child zone `dmz2-client9`
-
-The configuration tree could look like this:
-
- # tree /etc/icinga2/zones.d
- /etc/icinga2/zones.d
- ├── dmz1-checker
- │ └── health.conf
- ├── dmz1-client1
- │ └── hosts.conf
- ├── dmz1-client2
- │ └── hosts.conf
- ├── dmz2-checker
- │ └── health.conf
- ├── dmz2-client9
- │ └── hosts.conf
- ├── global-templates
- │ ├── apply_notifications.conf
- │ ├── apply_services.conf
- │ ├── commands.conf
- │ ├── groups.conf
- │ ├── templates.conf
- │ └── users.conf
- ├── ha-master
- │ └── health.conf
- └── README
-
- 7 directories, 13 files
-
-If you prefer a different naming schema for directories or file names, go for it. If you
-are unsure about the best method, join the [support channels](1-about.md#support) and discuss
-with the community.
-
-If you are planning to synchronize local service health checks inside a zone, look into the
-[command endpoint](13-distributed-monitoring-ha.md#cluster-health-check-command-endpoint)
-explainations.
-
-[Apply rules](3-monitoring-basics.md#using-apply) in zone directories underneath `zones.d`
-also match against objects defined outside of that particular zone directory.
-
-To work around this issue you can use an `assign where` rule to limit the apply rule to
-a specific zone:
-
- assign where host.zone == "dmz1-checker"
-
-## <a id="cluster-health-check"></a> Cluster Health Check
-
-The Icinga 2 [ITL](7-icinga-template-library.md#icinga-template-library) provides
-an internal check command checking all configured `EndPoints` in the cluster setup.
-The check result will become critical if one or more configured nodes are not connected.
-
-Example:
-
- object Host "icinga2a" {
- display_name = "Health Checks on icinga2a"
-
- address = "192.168.33.10"
- check_command = "hostalive"
- }
-
- object Service "cluster" {
- check_command = "cluster"
- check_interval = 5s
- retry_interval = 1s
-
- host_name = "icinga2a"
- }
-
-Each cluster node should execute its own local cluster health check to
-get an idea about network related connection problems from different
-points of view.
-
-Additionally you can monitor the connection from the local zone to the remote
-connected zones.
-
-Example for the `checker` zone checking the connection to the `master` zone:
-
- object Service "cluster-zone-master" {
- check_command = "cluster-zone"
- check_interval = 5s
- retry_interval = 1s
- vars.cluster_zone = "master"
-
- host_name = "icinga2b"
- }
-
-## <a id="cluster-health-check-command-endpoint"></a> Cluster Health Check with Command Endpoints
-
-If you are planning to sync the zone configuration inside a [High-Availability]()
-cluster zone, you can also use the `command_endpoint` object attribute to
-pin host/service checks to a specific endpoint inside the same zone.
-
-This requires the `accept_commands` setting inside the [ApiListener](13-distributed-monitoring-ha.md#configure-apilistener-object)
-object set to `true` similar to the [remote client command execution bridge](11-icinga2-client.md#icinga2-client-configuration-command-bridge)
-setup.
-
-Make sure to set `command_endpoint` to the correct endpoint instance.
-The example below assumes that the endpoint name is the same as the
-host name configured for health checks. If it differs, define a host
-custom attribute providing [this information](11-icinga2-client.md#icinga2-client-configuration-command-bridge-master-config).
-
- apply Service "cluster-ha" {
- check_command = "cluster"
- check_interval = 5s
- retry_interval = 1s
- /* make sure host.name is the same as endpoint name */
- command_endpoint = host.name
-
- assign where regex("^icinga2[a|b]", host.name)
- }
-
-
-## <a id="cluster-scenarios"></a> Cluster Scenarios
-
-All cluster nodes are full-featured Icinga 2 instances. You only need to enabled
-the features for their role (for example, a `Checker` node only requires the `checker`
-feature enabled, but not `notification` or `ido-mysql` features).
-
-> **Tip**
->
-> There's a [Vagrant demo setup](https://github.com/Icinga/icinga-vagrant/tree/master/icinga2x-cluster)
-> available featuring a two node cluster showcasing several aspects (config sync,
-> remote command execution, etc.).
-
-### <a id="cluster-scenarios-master-satellite-clients"></a> Cluster with Master, Satellites and Remote Clients
-
-You can combine "classic" cluster scenarios from HA to Master-Checker with the
-Icinga 2 Remote Client modes. Each instance plays a certain role in that picture.
-
-Imagine the following scenario:
-
-* The master zone acts as High-Availability zone
-* Remote satellite zones execute local checks and report them to the master
-* All satellites query remote clients and receive check results (which they also replay to the master)
-* All involved nodes share the same configuration logic: zones, endpoints, apilisteners
-
-You'll need to think about the following:
-
-* Deploy the entire configuration from the master to satellites and cascading remote clients? ("top down")
-* Use local client configuration instead and report the inventory to satellites and cascading to the master? ("bottom up")
-* Combine that with command execution bridges on remote clients and also satellites
-
-In case you want to use [CSR Auto-Signing](11-icinga2-client.md#csr-autosigning-requirements) in
-a three level cluster you'll need to ensure that the clients can connect to the master node once.
-The setup wizard can still be configured to connect to the satellite node following the example
-below.
-
- # icinga2 node wizard
- ...
- Please specify the master endpoint(s) this node should connect to:
- Master Common Name (CN from your master setup): icinga2-satellite1.localdomain
- Please fill out the master connection information:
- Master endpoint host (optional, your master's IP address or FQDN): icinga2-satellite1.localdomain
- ...
- Please specify the master connection for CSR auto-signing (defaults to master endpoint host):
- Host [icinga2-satellite1.localdomain]: icinga2-master1.localdomain
-
-Alternatively you can copy the CA director from your master in `/var/lib/icinga2/ca` to your satellites
-and connect to them using the client setup wizards.
-
-
-### <a id="cluster-scenarios-security"></a> Security in Cluster Scenarios
-
-While there are certain capabilities to ensure the safe communication between all
-nodes (firewalls, policies, software hardening, etc.) the Icinga 2 cluster also provides
-additional security itself:
-
-* [SSL certificates](13-distributed-monitoring-ha.md#manual-certificate-generation) are mandatory for cluster communication.
-* Child zones only receive event updates (check results, commands, etc.) for their configured updates.
-* Zones cannot influence/interfere other zones. Each checked object is assigned to only one zone.
-* All nodes in a zone trust each other.
-* [Configuration sync](13-distributed-monitoring-ha.md#zone-config-sync-permissions) is disabled by default.
-
-### <a id="cluster-scenarios-features"></a> Features in Cluster Zones
-
-Each cluster zone may use all available features. If you have multiple locations
-or departments, they may write to their local database, or populate graphite.
-Even further all commands are distributed amongst connected nodes. For example, you could
-re-schedule a check or acknowledge a problem on the master, and it gets replicated to the
-actual slave checker node.
-
-> **Note**
->
-> All features must be same on all endpoints inside an [HA zone](13-distributed-monitoring-ha.md#cluster-scenarios-high-availability).
-> There are additional [High-Availability-enabled features](13-distributed-monitoring-ha.md#high-availability-features) available.
-
-### <a id="cluster-scenarios-distributed-zones"></a> Distributed Zones
-
-That scenario fits if your instances are spread over the globe and they all report
-to a master instance. Their network connection only works towards the master master
-(or the master is able to connect, depending on firewall policies) which means
-remote instances won't see each/connect to each other.
-
-All events (check results, downtimes, comments, etc.) are synced to the master node,
-but the remote nodes can still run local features such as a web interface, reporting,
-graphing, etc. in their own specified zone.
-
-Imagine the following example with a master node in Nuremberg, and two remote DMZ
-based instances in Berlin and Vienna. Additonally you'll specify
-[global templates](13-distributed-monitoring-ha.md#zone-global-config-templates) available in all zones.
-
-The configuration tree on the master instance `nuremberg` could look like this:
-
- zones.d
- global-templates/
- templates.conf
- groups.conf
- nuremberg/
- local.conf
- berlin/
- hosts.conf
- vienna/
- hosts.conf
-
-The configuration deployment will take care of automatically synchronising
-the child zone configuration:
-
-* The master node sends `zones.d/berlin` to the `berlin` child zone.
-* The master node sends `zones.d/vienna` to the `vienna` child zone.
-* The master node sends `zones.d/global-templates` to the `vienna` and `berlin` child zones.
-
-The endpoint configuration would look like:
-
- object Endpoint "nuremberg-master" {
- host = "nuremberg.icinga.org"
- }
-
- object Endpoint "berlin-satellite" {
- host = "berlin.icinga.org"
- }
-
- object Endpoint "vienna-satellite" {
- host = "vienna.icinga.org"
- }
-
-The zones would look like:
-
- object Zone "nuremberg" {
- endpoints = [ "nuremberg-master" ]
- }
-
- object Zone "berlin" {
- endpoints = [ "berlin-satellite" ]
- parent = "nuremberg"
- }
-
- object Zone "vienna" {
- endpoints = [ "vienna-satellite" ]
- parent = "nuremberg"
- }
-
- object Zone "global-templates" {
- global = true
- }
-
-The `nuremberg-master` zone will only execute local checks, and receive
-check results from the satellite nodes in the zones `berlin` and `vienna`.
-
-> **Note**
->
-> The child zones `berlin` and `vienna` will get their configuration synchronised
-> from the configuration master 'nuremberg'. The endpoints in the child
-> zones **must not** have their `zones.d` directory populated if this endpoint
-> [accepts synced configuration](13-distributed-monitoring-ha.md#zone-config-sync-permissions).
-
-### <a id="cluster-scenarios-load-distribution"></a> Load Distribution
-
-If you are planning to off-load the checks to a defined set of remote workers,
-you can achieve that by:
-
-* Deploying the configuration on all nodes.
-* Let Icinga 2 distribute the load amongst all available nodes.
-
-That way all remote check instances will receive the same configuration
-but only execute their part. The master instance located in the `master` zone
-can also execute checks, but you may also disable the `Checker` feature.
-
-Configuration on the master node:
-
- zones.d/
- global-templates/
- master/
- checker/
-
-If you are planning to have some checks executed by a specific set of checker nodes,
-you have to define additional zones and define these check objects there.
-
-Endpoints:
-
- object Endpoint "master-node" {
- host = "master.icinga.org"
- }
-
- object Endpoint "checker1-node" {
- host = "checker1.icinga.org"
- }
-
- object Endpoint "checker2-node" {
- host = "checker2.icinga.org"
- }
-
-
-Zones:
-
- object Zone "master" {
- endpoints = [ "master-node" ]
- }
-
- object Zone "checker" {
- endpoints = [ "checker1-node", "checker2-node" ]
- parent = "master"
- }
-
- object Zone "global-templates" {
- global = true
- }
-
-> **Note**
->
-> The child zones `checker` will get its configuration synchronised
-> from the configuration master 'master'. The endpoints in the child
-> zone **must not** have their `zones.d` directory populated if this endpoint
-> [accepts synced configuration](13-distributed-monitoring-ha.md#zone-config-sync-permissions).
-
-### <a id="cluster-scenarios-high-availability"></a> Cluster High Availability
-
-High availability with Icinga 2 is possible by putting multiple nodes into
-a dedicated [zone](13-distributed-monitoring-ha.md#configure-cluster-zones). All nodes will elect one
-active master, and retry an election once the current active master is down.
-
-Selected features provide advanced [HA functionality](13-distributed-monitoring-ha.md#high-availability-features).
-Checks and notifications are load-balanced between nodes in the high availability
-zone.
-
-Connections from other zones will be accepted by all active and passive nodes
-but all are forwarded to the current active master dealing with the check results,
-commands, etc.
-
- object Zone "config-ha-master" {
- endpoints = [ "icinga2a", "icinga2b", "icinga2c" ]
- }
-
-Two or more nodes in a high availability setup require an [initial cluster sync](13-distributed-monitoring-ha.md#initial-cluster-sync).
-
-> **Note**
->
-> Keep in mind that **only one node acts as configuration master** having the
-> configuration files in the `zones.d` directory. All other nodes **must not**
-> have that directory populated. Instead they are required to
-> [accept synced configuration](13-distributed-monitoring-ha.md#zone-config-sync-permissions).
-> Details in the [Configuration Sync Chapter](13-distributed-monitoring-ha.md#cluster-zone-config-sync).
-
-### <a id="cluster-scenarios-multiple-hierarchies"></a> Multiple Hierarchies
-
-Your master zone collects all check results for reporting and graphing and also
-does some sort of additional notifications.
-The customers got their own instances in their local DMZ zones. They are limited to read/write
-only their services, but replicate all events back to the master instance.
-Within each DMZ there are additional check instances also serving interfaces for local
-departments. The customers instances will collect all results, but also send them back to
-your master instance.
-Additionally the customers instance on the second level in the middle prohibits you from
-sending commands to the subjacent department nodes. You're only allowed to receive the
-results, and a subset of each customers configuration too.
-
-Your master zone will generate global reports, aggregate alert notifications, and check
-additional dependencies (for example, the customers internet uplink and bandwidth usage).
-
-The customers zone instances will only check a subset of local services and delegate the rest
-to each department. Even though it acts as configuration master with a master dashboard
-for all departments managing their configuration tree which is then deployed to all
-department instances. Furthermore the master NOC is able to see what's going on.
-
-The instances in the departments will serve a local interface, and allow the administrators
-to reschedule checks or acknowledge problems for their services.
-
-
-## <a id="high-availability-features"></a> High Availability for Icinga 2 features
-
-All nodes in the same zone require the same features enabled for High Availability (HA)
-amongst them.
-
-By default the following features provide advanced HA functionality:
-
-* [Checks](13-distributed-monitoring-ha.md#high-availability-checks) (load balanced, automated failover)
-* [Notifications](13-distributed-monitoring-ha.md#high-availability-notifications) (load balanced, automated failover)
-* [DB IDO](13-distributed-monitoring-ha.md#high-availability-db-ido) (Run-Once, automated failover)
-
-### <a id="high-availability-checks"></a> High Availability with Checks
-
-All instances within the same zone (e.g. the `master` zone as HA cluster) must
-have the `checker` feature enabled.
-
-Example:
-
- # icinga2 feature enable checker
-
-All nodes in the same zone load-balance the check execution. When one instance shuts down
-the other nodes will automatically take over the reamining checks.
-
-### <a id="high-availability-notifications"></a> High Availability with Notifications
-
-All instances within the same zone (e.g. the `master` zone as HA cluster) must
-have the `notification` feature enabled.
-
-Example:
-
- # icinga2 feature enable notification
-
-Notifications are load balanced amongst all nodes in a zone. By default this functionality
-is enabled.
-If your nodes should notify independent from any other nodes (this will cause
-duplicated notifications if not properly handled!), you can set `enable_ha = false`
-in the [NotificationComponent](6-object-types.md#objecttype-notificationcomponent) feature.
-
-### <a id="high-availability-db-ido"></a> High Availability with DB IDO
-
-All instances within the same zone (e.g. the `master` zone as HA cluster) must
-have the DB IDO feature enabled.
-
-Example DB IDO MySQL:
-
- # icinga2 feature enable ido-mysql
-
-By default the DB IDO feature only runs on one node. All other nodes in the same zone disable
-the active IDO database connection at runtime. The node with the active DB IDO connection is
-not necessarily the zone master.
-
-> **Note**
->
-> The DB IDO HA feature can be disabled by setting the `enable_ha` attribute to `false`
-> for the [IdoMysqlConnection](6-object-types.md#objecttype-idomysqlconnection) or
-> [IdoPgsqlConnection](6-object-types.md#objecttype-idopgsqlconnection) object on **all** nodes in the
-> **same** zone.
->
-> All endpoints will enable the DB IDO feature and connect to the configured
-> database and dump configuration, status and historical data on their own.
-
-If the instance with the active DB IDO connection dies, the HA functionality will
-automatically elect a new DB IDO master.
-
-The DB IDO feature will try to determine which cluster endpoint is currently writing
-to the database and bail out if another endpoint is active. You can manually verify that
-by running the following query:
-
- icinga=> SELECT status_update_time, endpoint_name FROM icinga_programstatus;
- status_update_time | endpoint_name
- ------------------------+---------------
- 2014-08-15 15:52:26+02 | icinga2a
- (1 Zeile)
-
-This is useful when the cluster connection between endpoints breaks, and prevents
-data duplication in split-brain-scenarios. The failover timeout can be set for the
-`failover_timeout` attribute, but not lower than 60 seconds.
-
-
-## <a id="cluster-add-node"></a> Add a new cluster endpoint
-
-These steps are required for integrating a new cluster endpoint:
-
-* generate a new [SSL client certificate](13-distributed-monitoring-ha.md#manual-certificate-generation)
-* identify its location in the zones
-* update the `zones.conf` file on each involved node ([endpoint](13-distributed-monitoring-ha.md#configure-cluster-endpoints), [zones](13-distributed-monitoring-ha.md#configure-cluster-zones))
- * a new slave zone node requires updates for the master and slave zones
- * verify if this endpoints requires [configuration synchronisation](13-distributed-monitoring-ha.md#cluster-zone-config-sync) enabled
-* if the node requires the existing zone history: [initial cluster sync](13-distributed-monitoring-ha.md#initial-cluster-sync)
-* add a [cluster health check](13-distributed-monitoring-ha.md#cluster-health-check)
-
-### <a id="initial-cluster-sync"></a> Initial Cluster Sync
-
-In order to make sure that all of your cluster nodes have the same state you will
-have to pick one of the nodes as your initial "master" and copy its state file
-to all the other nodes.
-
-You can find the state file in `/var/lib/icinga2/icinga2.state`. Before copying
-the state file you should make sure that all your cluster nodes are properly shut
-down.
-
-
-## <a id="host-multiple-cluster-nodes"></a> Host With Multiple Cluster Nodes
-
-Special scenarios might require multiple cluster nodes running on a single host.
-By default Icinga 2 and its features will place their runtime data below the prefix
-`LocalStateDir`. By default packages will set that path to `/var`.
-You can either set that variable as constant configuration
-definition in [icinga2.conf](4-configuring-icinga-2.md#icinga2-conf) or pass it as runtime variable to
-the Icinga 2 daemon.
-
- # icinga2 -c /etc/icinga2/node1/icinga2.conf -DLocalStateDir=/opt/node1/var
Details on the installation can be found in the [Configuring DB IDO](2-getting-started.md#configuring-db-ido-mysql)
chapter. Details on the configuration can be found in the
-[IdoMysqlConnection](6-object-types.md#objecttype-idomysqlconnection) and
-[IdoPgsqlConnection](6-object-types.md#objecttype-idopgsqlconnection)
+[IdoMysqlConnection](9-object-types.md#objecttype-idomysqlconnection) and
+[IdoPgsqlConnection](9-object-types.md#objecttype-idopgsqlconnection)
object configuration documentation.
-The DB IDO feature supports [High Availability](13-distributed-monitoring-ha.md#high-availability-db-ido) in
+The DB IDO feature supports [High Availability](6-distributed-monitoring.md#distributed-monitoring-high-availability-db-ido) in
the Icinga 2 cluster.
The following example query checks the health of the current Icinga 2 instance
> **Tip**
>
-> Use [check plugins](14-addons-plugins.md#plugins) to monitor the backend.
+> Use [check plugins](5-service-monitoring.md#service-monitoring-plugins) to monitor the backend.
Replace the `default` string with your instance name if different.
(1 Zeile)
-A detailed list on the available table attributes can be found in the [DB IDO Schema documentation](24-appendix.md#schema-db-ido).
+A detailed list on the available table attributes can be found in the [DB IDO Schema documentation](23-appendix.md#schema-db-ido).
## <a id="external-commands"></a> External Commands
Oct 17 15:01:25 icinga-server icinga2: Executing external command: [1382014885] SCHEDULE_FORCED_SVC_CHECK;localhost;ping4;1382014885
Oct 17 15:01:25 icinga-server icinga2: Rescheduling next check for service 'ping4'
-A list of currently supported external commands can be found [here](24-appendix.md#external-commands-list-detail).
+A list of currently supported external commands can be found [here](23-appendix.md#external-commands-list-detail).
Detailed information on the commands and their required parameters can be found
on the [Icinga 1.x documentation](http://docs.icinga.org/latest/en/extcommands2.html).
store them in their backends. These tools usually generate graphs for historical
reporting and trending.
-Well-known addons processing Icinga performance data are [PNP4Nagios](14-addons-plugins.md#addons-graphing-pnp),
-[Graphite](14-addons-plugins.md#addons-graphing-graphite) or [OpenTSDB](15-features.md#opentsdb-writer).
+Well-known addons processing Icinga performance data are [PNP4Nagios](13-addons.md#addons-graphing-pnp),
+[Graphite](13-addons.md#addons-graphing-graphite) or [OpenTSDB](14-features.md#opentsdb-writer).
### <a id="writing-performance-data-files"></a> Writing Performance Data Files
PNP4Nagios and Graphios use performance data collector daemons to fetch
the current performance files for their backend updates.
-Therefore the Icinga 2 [PerfdataWriter](6-object-types.md#objecttype-perfdatawriter)
+Therefore the Icinga 2 [PerfdataWriter](9-object-types.md#objecttype-perfdatawriter)
feature allows you to define the output template format for host and services helped
with Icinga 2 runtime vars.
### <a id="graphite-carbon-cache-writer"></a> Graphite Carbon Cache Writer
-While there are some [Graphite](14-addons-plugins.md#addons-graphing-graphite)
+While there are some [Graphite](13-addons.md#addons-graphing-graphite)
collector scripts and daemons like Graphios available for Icinga 1.x it's more
reasonable to directly process the check and plugin performance
in memory in Icinga 2. Once there are new metrics available, Icinga 2 will directly
# icinga2 feature enable graphite
-By default the [GraphiteWriter](6-object-types.md#objecttype-graphitewriter) feature
+By default the [GraphiteWriter](9-object-types.md#objecttype-graphitewriter) feature
expects the Graphite Carbon Cache to listen at `127.0.0.1` on TCP port `2003`.
#### <a id="graphite-carbon-cache-writer-schema"></a> Current Graphite Schema
`service_name_template` configuration attributes.
The example below uses [runtime macros](3-monitoring-basics.md#runtime-macros) and a
-[global constant](18-language-reference.md#constants) named `GraphiteEnv`. The constant name
+[global constant](17-language-reference.md#constants) named `GraphiteEnv`. The constant name
is freely definable and should be put in the [constants.conf](4-configuring-icinga-2.md#constants-conf) file.
const GraphiteEnv = "icinga.env1"
# icinga2 feature enable influxdb
-By default the [InfluxdbWriter](6-object-types.md#objecttype-influxdbwriter) feature
+By default the [InfluxdbWriter](9-object-types.md#objecttype-influxdbwriter) feature
expects the InfluxDB daemon to listen at `127.0.0.1` on port `8086`.
-More configuration details can be found [here](6-object-types.md#objecttype-influxdbwriter).
+More configuration details can be found [here](9-object-types.md#objecttype-influxdbwriter).
### <a id="gelfwriter"></a> GELF Writer
Livestatus.
Details on the available tables and attributes with Icinga 2 can be found
-in the [Livestatus Schema](24-appendix.md#schema-livestatus) section.
+in the [Livestatus Schema](23-appendix.md#schema-livestatus) section.
You can enable Livestatus using icinga2 feature enable:
* Unix socket (default)
* TCP socket
-Details on the configuration can be found in the [LivestatusListener](6-object-types.md#objecttype-livestatuslistener)
+Details on the configuration can be found in the [LivestatusListener](9-object-types.md#objecttype-livestatuslistener)
object configuration.
### <a id="livestatus-get-queries"></a> Livestatus GET Queries
### <a id="livestatus-command-queries"></a> Livestatus COMMAND Queries
-A list of available external commands and their parameters can be found [here](24-appendix.md#external-commands-list-detail)
+A list of available external commands and their parameters can be found [here](23-appendix.md#external-commands-list-detail)
$ echo -e 'COMMAND <externalcommandstring>' | netcat 127.0.0.1 6558
downtimes | services | status attributes
timeperiods | | name and is inside flag
endpoints | | config and status attributes
- log | services, hosts, contacts, commands | parses [compatlog](6-object-types.md#objecttype-compatlogger) and shows log attributes
- statehist | hosts, services | parses [compatlog](6-object-types.md#objecttype-compatlogger) and aggregates state change attributes
+ log | services, hosts, contacts, commands | parses [compatlog](9-object-types.md#objecttype-compatlogger) and shows log attributes
+ statehist | hosts, services | parses [compatlog](9-object-types.md#objecttype-compatlogger) and aggregates state change attributes
hostsbygroup | hostgroups | host attributes grouped by hostgroup and its attributes
servicesbygroup | servicegroups | service attributes grouped by servicegroup and its attributes
servicesbyhostgroup | hostgroups | service attributes grouped by hostgroup and its attributes
The `commands` table is populated with `CheckCommand`, `EventCommand` and `NotificationCommand` objects.
-A detailed list on the available table attributes can be found in the [Livestatus Schema documentation](24-appendix.md#schema-livestatus).
+A detailed list on the available table attributes can be found in the [Livestatus Schema documentation](23-appendix.md#schema-livestatus).
## <a id="status-data"></a> Status Data Files
These logs are not only used for informational representation in
external web interfaces parsing the logs, but also to generate
SLA reports and trends in Icinga 1.x Classic UI. Furthermore the
-[Livestatus](15-features.md#setting-up-livestatus) feature uses these logs for answering queries to
+[Livestatus](14-features.md#setting-up-livestatus) feature uses these logs for answering queries to
historical tables.
The `CompatLogger` object can be enabled with
* How was Icinga 2 installed (and which repository in case) and which distribution are you using
* Provide complete configuration snippets explaining your problem in detail
* If the check command failed, what's the output of your manual plugin tests?
-* In case of [debugging](21-development.md#development) Icinga 2, the full back traces and outputs
+* In case of [debugging](20-development.md#development) Icinga 2, the full back traces and outputs
## <a id="troubleshooting-enable-debug-output"></a> Enable Debug Output
> **Tip**
>
-> Use the Icinga 2 API to access [config objects at runtime](9-icinga2-api.md#icinga2-api-config-objects) directly.
+> Use the Icinga 2 API to access [config objects at runtime](12-icinga2-api.md#icinga2-api-config-objects) directly.
-That way you can also identify which objects have been created from your [apply rules](18-language-reference.md#apply).
+That way you can also identify which objects have been created from your [apply rules](17-language-reference.md#apply).
# icinga2 object list
## <a id="check-command-definitions"></a> Where are the check command definitions?
-Icinga 2 features a number of built-in [check command definitions](7-icinga-template-library.md#plugin-check-commands) which are
+Icinga 2 features a number of built-in [check command definitions](10-icinga-template-library.md#plugin-check-commands) which are
included using
include <itl>
### <a id="checks-executed-command"></a> Executed Command for Checks
-* Use the Icinga 2 API to [query](9-icinga2-api.md#icinga2-api-config-objects-query) host/service objects
+* Use the Icinga 2 API to [query](12-icinga2-api.md#icinga2-api-config-objects-query) host/service objects
for their check result containing the executed shell command.
-* Use the Icinga 2 [console cli command](8-cli-commands.md#cli-command-console)
+* Use the Icinga 2 [console cli command](11-cli-commands.md#cli-command-console)
to fetch the checkable object, its check result and the executed shell command.
-* Alternatively enable the [debug log](16-troubleshooting.md#troubleshooting-enable-debug-output) and look for the executed command.
+* Alternatively enable the [debug log](15-troubleshooting.md#troubleshooting-enable-debug-output) and look for the executed command.
### <a id="checks-not-executed"></a> Checks are not executed
-* Check the [debug log](16-troubleshooting.md#troubleshooting-enable-debug-output) to see if the check command gets executed.
+* Check the [debug log](15-troubleshooting.md#troubleshooting-enable-debug-output) to see if the check command gets executed.
* Verify that failed depedencies do not prevent command execution.
* Make sure that the plugin is executable by the Icinga 2 user (run a manual test).
-* Make sure the [checker](8-cli-commands.md#enable-features) feature is enabled.
-* Use the Icinga 2 API [event streams](9-icinga2-api.md#icinga2-api-event-streams) to receive live check result streams.
+* Make sure the [checker](11-cli-commands.md#enable-features) feature is enabled.
+* Use the Icinga 2 API [event streams](12-icinga2-api.md#icinga2-api-event-streams) to receive live check result streams.
Examples:
* Do the notification attributes `states`, `types`, `period` match the notification conditions?
* Do the user attributes `states`, `types`, `period` match the notification conditions?
* Are there any notification `begin` and `end` times configured?
-* Make sure the [notification](8-cli-commands.md#enable-features) feature is enabled.
+* Make sure the [notification](11-cli-commands.md#enable-features) feature is enabled.
* Does the referenced NotificationCommand work when executed as Icinga user on the shell?
If notifications are to be sent via mail, make sure that the mail program specified inside the
-[NotificationCommand object](6-object-types.md#objecttype-notificationcommand) exists.
+[NotificationCommand object](9-object-types.md#objecttype-notificationcommand) exists.
The name and location depends on the distribution so the preconfigured setting might have to be
changed on your system.
# icinga2 feature enable notification
The feature 'notification' is already enabled.
-You can use the Icinga 2 API [event streams](9-icinga2-api.md#icinga2-api-event-streams) to receive live notification streams:
+You can use the Icinga 2 API [event streams](12-icinga2-api.md#icinga2-api-event-streams) to receive live notification streams:
$ curl -k -s -u root:icinga -X POST 'https://localhost:5665/v1/events?queue=debugnotifications&types=Notification'
## <a id="configuration-ignored"></a> Configuration is ignored
-* Make sure that the line(s) are not [commented out](18-language-reference.md#comments) (starting with `//` or `#`, or
+* Make sure that the line(s) are not [commented out](17-language-reference.md#comments) (starting with `//` or `#`, or
encapsulated by `/* ... */`).
* Is the configuration file included in [icinga2.conf](4-configuring-icinga-2.md#icinga2-conf)?
## <a id="configuration-attribute-inheritance"></a> Configuration attributes are inherited from
-Icinga 2 allows you to import templates using the [import](18-language-reference.md#template-imports) keyword. If these templates
+Icinga 2 allows you to import templates using the [import](17-language-reference.md#template-imports) keyword. If these templates
contain additional attributes, your objects will automatically inherit them. You can override
or modify these attributes in the current object.
## <a id="troubleshooting-cluster"></a> Cluster and Clients Troubleshooting
-This applies to anything using the cluster protocol:
+This applies to any Icinga 2 node in a [distributed monitoring setup](6-distributed-monitoring.md#distributed-monitoring-scenarios).
-* [Distributed and High-Availability](13-distributed-monitoring-ha.md#distributed-monitoring-high-availability) scenarios
-* [Remote client](11-icinga2-client.md#icinga2-client-scenarios) scenarios
-
-You should configure the [cluster health checks](13-distributed-monitoring-ha.md#cluster-health-check) if you haven't
+You should configure the [cluster health checks](6-distributed-monitoring.md#distributed-monitoring-health-checks) if you haven't
done so already.
> **Note**
---
...
-If the connection attempt fails or your CA does not match, [verify the master and client certificates](16-troubleshooting.md#troubleshooting-cluster-ssl-certificate-verification).
+If the connection attempt fails or your CA does not match, [verify the master and client certificates](15-troubleshooting.md#troubleshooting-cluster-ssl-certificate-verification).
#### <a id="troubleshooting-cluster-unauthenticated-clients"></a> Cluster Troubleshooting Unauthenticated Clients
-Unauthenticated nodes are able to connect required by the
-[CSR auto-signing](11-icinga2-client.md#csr-autosigning-requirements) functionality.
+Unauthenticated nodes are able to connect. This is required for client setups.
Master:
[2015-07-13 18:29:26 +1000] notice/ApiEvents: Discarding 'execute command' message from 'icinga-master': Invalid endpoint origin (client not allowed).
-If these messages do not go away, make sure to [verify the master and client certificates](16-troubleshooting.md#troubleshooting-cluster-ssl-certificate-verification).
+If these messages do not go away, make sure to [verify the master and client certificates](15-troubleshooting.md#troubleshooting-cluster-ssl-certificate-verification).
#### <a id="troubleshooting-cluster-ssl-certificate-verification"></a> Cluster Troubleshooting SSL Certificate Verification
will be disconnected. If the connection can't be re-established between endpoints in the same HA zone,
they remain in a Split-Brain-mode and history may differ.
-Although the Icinga 2 cluster protocol stores historical events in a [replay log](16-troubleshooting.md#troubleshooting-cluster-replay-log)
+Although the Icinga 2 cluster protocol stores historical events in a [replay log](15-troubleshooting.md#troubleshooting-cluster-replay-log)
for later synchronisation, you should make sure to check why the network connection failed.
### <a id="troubleshooting-cluster-command-endpoint-errors"></a> Cluster Troubleshooting Command Endpoint Errors
-Command endpoints can be used for clients acting as [remote command execution bridge](11-icinga2-client.md#icinga2-client-configuration-command-bridge)
-as well as inside an [High-Availability cluster](13-distributed-monitoring-ha.md#distributed-monitoring-high-availability).
+Command endpoints can be used [for clients](6-distributed-monitoring.md#distributed-monitoring-top-down-command-endpoint)
+as well as inside an [High-Availability cluster](6-distributed-monitoring.md#distributed-monitoring-scenarios).
There is no cli command for manually executing the check, but you can verify
the following (e.g. by invoking a forced check from the web interface):
* `/var/log/icinga2/icinga2.log` contains connection and execution errors.
- * The ApiListener is not enabled to [accept commands](11-icinga2-client.md#icinga2-client-configuration-command-bridge).
+ * The ApiListener is not enabled to [accept commands](6-distributed-monitoring.md#distributed-monitoring-top-down-command-endpoint).
* `CheckCommand` definition not found on the remote client.
* Referenced check plugin not found on the remote client.
* Runtime warnings and errors, e.g. unresolved runtime macros or configuration problems.
* Specific error messages are also populated into `UNKNOWN` check results including a detailed error message in their output.
* Verify the `check_source` object attribute. This is populated by the node executing the check.
-* More verbose logs are found inside the [debug log](16-troubleshooting.md#troubleshooting-enable-debug-output).
+* More verbose logs are found inside the [debug log](15-troubleshooting.md#troubleshooting-enable-debug-output).
-* Use the Icinga 2 API [event streams](9-icinga2-api.md#icinga2-api-event-streams) to receive live check result streams.
+* Use the Icinga 2 API [event streams](12-icinga2-api.md#icinga2-api-event-streams) to receive live check result streams.
Fetch all check result events matching the `event.service` name `remote-client`:
** The master syncs the configuration to `/var/lib/icinga2/api/zones/` during startup and only syncs valid configuration to the other nodes.
** The other nodes receive the configuration into `/var/lib/icinga2/api/zones/`.
* The `icinga2.log` log file in `/var/log/icinga2` will indicate whether this ApiListener
-[accepts config](13-distributed-monitoring-ha.md#zone-config-sync-permissions), or not.
+[accepts config](6-distributed-monitoring.md#distributed-monitoring-top-down-config-sync), or not.
-Verify the object's [version](6-object-types.md#object-types) attribute on all nodes to
+Verify the object's [version](9-object-types.md#object-types) attribute on all nodes to
check whether the config update and reload was succesful or not.
### <a id="troubleshooting-cluster-check-results"></a> Cluster Troubleshooting Overdue Check Results
(satellite, clients, etc.), make sure to check whether the client sending in events
is allowed to do so.
-The [cluster naming convention](13-distributed-monitoring-ha.md#cluster-naming-convention)
-applies. So, if there's a mismatch between your client node's endpoint name and its provided
+The [distributed monitoring conventions](6-distributed-monitoring.md#distributed-monitoring-conventions)
+apply. So, if there's a mismatch between your client node's endpoint name and its provided
certificate's CN, the master will deny all events.
> **Tip**
> [Icinga Web 2](2-getting-started.md#setting-up-icingaweb2) provides a dashboard view
> for overdue check results.
-Enable the [debug log](16-troubleshooting.md#troubleshooting-enable-debug-output) on the master
+Enable the [debug log](15-troubleshooting.md#troubleshooting-enable-debug-output) on the master
for more verbose insights.
-If the client cannot authenticate, it's a more general [problem](16-troubleshooting.md#troubleshooting-cluster-unauthenticated-clients).
+If the client cannot authenticate, it's a more general [problem](15-troubleshooting.md#troubleshooting-cluster-unauthenticated-clients).
The client's endpoint is not configured on nor trusted by the master node:
Check the following:
-* All clients are connected? (e.g. [cluster health check](13-distributed-monitoring-ha.md#cluster-health-check)).
-* Check your [connection](16-troubleshooting.md#troubleshooting-cluster-connection-errors) in general.
+* All clients are connected? (e.g. [cluster health check](6-distributed-monitoring.md#distributed-monitoring-health-checks)).
+* Check your [connection](15-troubleshooting.md#troubleshooting-cluster-connection-errors) in general.
* Does the log replay work, e.g. are all events processed and the directory gets cleared up over time?
-* Decrease the `log_duration` attribute value for that specific [endpoint](6-object-types.md#objecttype-endpoint).
+* Decrease the `log_duration` attribute value for that specific [endpoint](9-object-types.md#objecttype-endpoint).
check_interval = len(MyGroups) * 1m
}
-A list of available functions is available in the [Library Reference](19-library-reference.md#library-reference) chapter.
+A list of available functions is available in the [Library Reference](18-library-reference.md#library-reference) chapter.
## <a id="dictionary-operators"></a> Assignments
In this example the `assign where` condition is a boolean expression which is
evaluated for all objects of type `Host` and a new service with name "ping"
-is created for each matching host. [Expression operators](18-language-reference.md#expression-operators)
+is created for each matching host. [Expression operators](17-language-reference.md#expression-operators)
may be used in `assign where` conditions.
The `to` keyword and the target type may be omitted if there is only one target
## <a id="apply-for"></a> Apply For
-[Apply](18-language-reference.md#apply) rules can be extended with the
-[for loop](18-language-reference.md#for-loops) keyword.
+[Apply](17-language-reference.md#apply) rules can be extended with the
+[for loop](17-language-reference.md#for-loops) keyword.
apply Service "prefix-" for (key => value in host.vars.dictionary) to Host {
import "generic-service"
In this example the `assign where` condition is a boolean expression which is evaluated
for all objects of the type `Host`. Each matching host is added as member to the host group
with the name "linux-servers". Membership exclusion can be controlled using the `ignore where`
-condition. [Expression operators](18-language-reference.md#expression-operators) may be used in `assign where` and
+condition. [Expression operators](17-language-reference.md#expression-operators) may be used in `assign where` and
`ignore where` conditions.
Source Type | Variables
Non-empty dictionary | { key = "value" } | true
For a list of supported expression operators for `assign where` and `ignore where`
-statements, see [expression operators](18-language-reference.md#expression-operators).
+statements, see [expression operators](17-language-reference.md#expression-operators).
## <a id="comments"></a> Comments
config compiler to search the include search paths for the specified
file. By default $PREFIX/share/icinga2/include is included in the list of search
paths. Additional include search paths can be added using
-[command-line options](8-cli-commands.md#config-include-path).
+[command-line options](11-cli-commands.md#config-include-path).
Wildcards are not permitted when using angle brackets.
Dictionary | { a = 3 } | A dictionary.
Depending on which libraries are loaded additional types may become available. The `icinga`
-library implements a whole bunch of other [object types](6-object-types.md#object-types),
+library implements a whole bunch of other [object types](9-object-types.md#object-types),
e.g. Host, Service, CheckCommand, etc.
Each type has an associated type object which describes the type's semantics. These
keys(String.prototype)
Additional documentation on type methods is available in the
-[library reference](19-library-reference.md#library-reference).
+[library reference](18-library-reference.md#library-reference).
## <a id="location-information"></a> Location Information
random() | Returns a random value between 0 and RAND_MAX (as defined in stdlib.h).
log(value) | Writes a message to the log. Non-string values are converted to a JSON string.
log(severity, facility, value) | Writes a message to the log. `severity` can be one of `LogDebug`, `LogNotice`, `LogInformation`, `LogWarning`, and `LogCritical`. Non-string values are converted to a JSON string.
-typeof(value) | Returns the [Type](19-library-reference.md#type-type) object for a value.
+typeof(value) | Returns the [Type](18-library-reference.md#type-type) object for a value.
get_time() | Returns the current UNIX timestamp.
parse_performance_data(pd) | Parses a performance data string and returns an array describing the values.
dirname(path) | Returns the directory portion of the specified path.
function contains(str);
Returns `true` if the string `str` was found in the string. If the string
-was not found, `false` is returned. Use [find](19-library-reference.md#string-find)
+was not found, `false` is returned. Use [find](18-library-reference.md#string-find)
for getting the index instead.
Example:
## <a id="type-type"></a> Type type
-Inherits methods from the [Object type](19-library-reference.md#object-type).
+Inherits methods from the [Object type](18-library-reference.md#object-type).
The `Type` type provides information about the underlying type of an object or scalar value.
## <a id="array-type"></a> Array type
-Inherits methods from the [Object type](19-library-reference.md#object-type).
+Inherits methods from the [Object type](18-library-reference.md#object-type).
### <a id="array-add"></a> Array#add
## <a id="dictionary-type"></a> Dictionary type
-Inherits methods from the [Object type](19-library-reference.md#object-type).
+Inherits methods from the [Object type](18-library-reference.md#object-type).
### <a id="dictionary-shallow-clone"></a> Dictionary#shallow_clone
## <a id="scriptfunction-type"></a> Function type
-Inherits methods from the [Object type](19-library-reference.md#object-type).
+Inherits methods from the [Object type](18-library-reference.md#object-type).
### <a id="scriptfunction-call"></a> Function#call
## <a id="datetime-type"></a> DateTime type
-Inherits methods from the [Object type](19-library-reference.md#object-type).
+Inherits methods from the [Object type](18-library-reference.md#object-type).
### <a id="datetime-ctor"></a> DateTime constructor
# icinga2 daemon -X
-When an exception occurs or the [debugger](18-language-reference.md#breakpoints)
+When an exception occurs or the [debugger](17-language-reference.md#breakpoints)
keyword is encountered in a user script, Icinga 2 launches a console that
allows the user to debug the script.
* `mainlog` for writing the `icinga2.log` file
You can verify that by calling `icinga2 feature list`
-[CLI command](8-cli-commands.md#cli-command-feature) to see which features are
+[CLI command](11-cli-commands.md#cli-command-feature) to see which features are
enabled and disabled.
# icinga2 feature list
> **Note**
>
-> Please refer to the [plugins](14-addons-plugins.md#plugins) chapter for details about how to integrate
+> Please refer to the [service monitoring](5-service-monitoring.md#service-monitoring-plugins) chapter for details about how to integrate
> additional check plugins into your Icinga 2 setup.
## <a id="running-icinga2"></a> Running Icinga 2
Job for icinga2.service failed. See 'systemctl status icinga2.service' and 'journalctl -xn' for details.
If you're stuck with configuration errors, you can manually invoke the
-[configuration validation](8-cli-commands.md#config-validation).
+[configuration validation](11-cli-commands.md#config-validation).
### FreeBSD
update the database credentials in this file.
All available attributes are explained in the
-[IdoMysqlConnection object](6-object-types.md#objecttype-idomysqlconnection)
+[IdoMysqlConnection object](9-object-types.md#objecttype-idomysqlconnection)
chapter.
You can enable the `ido-mysql` feature configuration file using
the database credentials in this file.
All available attributes are explained in the
-[IdoPgsqlConnection object](6-object-types.md#objecttype-idopgsqlconnection)
+[IdoPgsqlConnection object](9-object-types.md#objecttype-idopgsqlconnection)
chapter.
You can enable the `ido-pgsql` feature configuration file using
A number of additional features are available in the form of addons. A list of
popular addons is available in the
-[Addons and Plugins](14-addons-plugins.md#addons-plugins) chapter.
+[Addons and Plugins](13-addons.md#addons) chapter.
+
+## <a id="install-backup"></a> Backup
+
+Ensure to include the following in your backups:
+
+* Configuration files in `/etc/icinga2`
+* Runtime files in `/var/lib/icinga2` (the master's CA is stored here as well)
+* Optional: IDO database backup
For a long-term migration of your configuration you should consider re-creating
your configuration based on the proposed Icinga 2 configuration paradigm.
-Please read the [next chapter](23-migrating-from-icinga-1x.md#differences-1x-2) to find out more about the differences
+Please read the [next chapter](22-migrating-from-icinga-1x.md#differences-1x-2) to find out more about the differences
between 1.x and 2.
### <a id="manual-config-migration-hints"></a> Manual Config Migration Hints
straight into a possible Icinga 2 format. If you found a different strategy, please
let us know!
-If you require in-depth explanations, please check the [next chapter](23-migrating-from-icinga-1x.md#differences-1x-2).
+If you require in-depth explanations, please check the [next chapter](22-migrating-from-icinga-1x.md#differences-1x-2).
#### <a id="manual-config-migration-hints-Intervals"></a> Manual Config Migration Hints for Intervals
hostgroup_members hg1
}
-This can be migrated to Icinga 2 and [using group assign](18-language-reference.md#group-assign). The additional nested hostgroup
+This can be migrated to Icinga 2 and [using group assign](17-language-reference.md#group-assign). The additional nested hostgroup
`hg1` is included into `hg2` with the `groups` attribute.
#### <a id="manual-config-migration-hints-runtime-macros"></a> Manual Config Migration Hints for Runtime Macros
-Runtime macros have been renamed. A detailed comparison table can be found [here](23-migrating-from-icinga-1x.md#differences-1x-2-runtime-macros).
+Runtime macros have been renamed. A detailed comparison table can be found [here](22-migrating-from-icinga-1x.md#differences-1x-2-runtime-macros).
For example, accessing the service check output looks like the following in Icinga 1.x:
#### <a id="manual-config-migration-hints-contacts-users"></a> Manual Config Migration Hints for Contacts (Users)
Contacts in Icinga 1.x act as users in Icinga 2, but do not have any notification commands specified.
-This migration part is explained in the [next chapter](23-migrating-from-icinga-1x.md#manual-config-migration-hints-notifications).
+This migration part is explained in the [next chapter](22-migrating-from-icinga-1x.md#manual-config-migration-hints-notifications).
define contact{
contact_name testconfig-user
email icinga@localhost
}
-The `service_notification_options` can be [mapped](23-migrating-from-icinga-1x.md#manual-config-migration-hints-notification-filters)
+The `service_notification_options` can be [mapped](22-migrating-from-icinga-1x.md#manual-config-migration-hints-notification-filters)
into generic `state` and `type` filters, if additional notification filtering is required. `alias` gets
renamed to `display_name`.
Convert the `notification_options` attribute from Icinga 1.x to Icinga 2 `states` and `types`. Details
-[here](23-migrating-from-icinga-1x.md#manual-config-migration-hints-notification-filters). Add the notification period.
+[here](22-migrating-from-icinga-1x.md#manual-config-migration-hints-notification-filters). Add the notification period.
states = [ OK, Warning, Critical ]
types = [ Recovery, Problem, Custom ]
assign where "hg_svcdep2" in host.groups
}
-Host dependencies are explained in the [next chapter](23-migrating-from-icinga-1x.md#manual-config-migration-hints-host-parents).
+Host dependencies are explained in the [next chapter](22-migrating-from-icinga-1x.md#manual-config-migration-hints-host-parents).
* Icinga 2 does not support any 1.x NEB addons for check load distribution
* If your current setup consists of instances distributing the check load, you should consider
-building a [load distribution](13-distributed-monitoring-ha.md#cluster-scenarios-load-distribution) setup with Icinga 2.
+building a [load distribution](6-distributed-monitoring.md#distributed-monitoring-scenarios) setup with Icinga 2.
* If your current setup includes active/passive clustering with external tools like Pacemaker/DRBD,
-consider the [High Availability](13-distributed-monitoring-ha.md#cluster-scenarios-high-availability) setup.
+consider the [High Availability](6-distributed-monitoring.md#distributed-monitoring-scenarios) setup.
* If you have build your own custom configuration deployment and check result collecting mechanism,
you should re-design your setup and re-evaluate your requirements, and how they may be fulfilled
using the Icinga 2 cluster capabilities.
### <a id="differences-1x-2-main-config"></a> Main Config File
In Icinga 1.x there are many global configuration settings available in `icinga.cfg`.
-Icinga 2 only uses a small set of [global constants](18-language-reference.md#constants) allowing
+Icinga 2 only uses a small set of [global constants](17-language-reference.md#constants) allowing
you to specify certain different setting such as the `NodeName` in a cluster scenario.
Aside from that, the [icinga2.conf](4-configuring-icinga-2.md#icinga2-conf) should take care of including
-global constants, enabled [features](8-cli-commands.md#enable-features) and the object configuration.
+global constants, enabled [features](11-cli-commands.md#enable-features) and the object configuration.
### <a id="differences-1x-2-include-files-dirs"></a> Include Files and Directories
const PluginDir = "/usr/lib/nagios/plugins"
-[Global macros](18-language-reference.md#constants) can only be defined once. Trying to modify a
+[Global macros](17-language-reference.md#constants) can only be defined once. Trying to modify a
global constant will result in an error.
### <a id="differences-1x-2-configuration-comments"></a> Configuration Comments
using the [apply](3-monitoring-basics.md#using-apply) keyword.
Direct object relations between a service and a host still allow you to use
-the `host_name` [Service](6-object-types.md#objecttype-service) object attribute.
+the `host_name` [Service](9-object-types.md#objecttype-service) object attribute.
### <a id="differences-1x-2-users"></a> Users
are separated from the command name using an exclamation mark (`!`).
Please check the migration hints for a detailed
-[migration example](23-migrating-from-icinga-1x.md#manual-config-migration-hints-check-command-arguments).
+[migration example](22-migrating-from-icinga-1x.md#manual-config-migration-hints-check-command-arguments).
> **Note**
>
requiring some command timeouts to be extended.
Icinga 2 allows you to specify the command timeout directly on the command. So,
-if your VMVware check plugin takes 15 minutes, [increase the timeout](6-object-types.md#objecttype-checkcommand)
+if your VMVware check plugin takes 15 minutes, [increase the timeout](9-object-types.md#objecttype-checkcommand)
accordingly.
For detailed examples on how to use the dependencies please check the [dependencies](3-monitoring-basics.md#dependencies)
chapter.
-Dependencies can be applied to hosts or services using the [apply rules](18-language-reference.md#apply).
+Dependencies can be applied to hosts or services using the [apply rules](17-language-reference.md#apply).
The `StatusDataWriter`, `IdoMysqlConnection` and `LivestatusListener` types
support the Icinga 1.x schema with dependencies and parent attributes for
* parent process continues with old configuration objects and the event scheduling
(doing checks, replicating cluster events, triggering alert notifications, etc.)
* validation NOT ok: child process terminates, parent process continues with old configuration state
-(this is **essential** for the [cluster config synchronisation](13-distributed-monitoring-ha.md#cluster-zone-config-sync))
+(this is **essential** for the [cluster config synchronisation](6-distributed-monitoring.md#distributed-monitoring-top-down-config-sync))
* validation ok: child process signals parent process to terminate and save its current state
(all events until now) into the icinga2 state file
* parent process shuts down writing icinga2.state file
and configuration distribution problems Icinga 1.x distributed monitoring currently suffers from.
Icinga 2 implements a new built-in
-[distributed monitoring architecture](13-distributed-monitoring-ha.md#distributed-monitoring-high-availability),
+[distributed monitoring architecture](6-distributed-monitoring.md#distributed-monitoring-scenarios),
including config and check distribution, IPv4/IPv6 support, SSL certificates and zone support for DMZ.
High Availability and load balancing are also part of the Icinga 2 Cluster feature, next to local replay
logs on connection loss ensuring that the event history is kept in sync.
This part of the Icinga 2 documentation provides an overview of all the basic
monitoring concepts you need to know to run Icinga 2.
Keep in mind these examples are made with a Linux server in mind. If you are
-using Windows, you will need to change the services accordingly. See the [ITL reference](7-icinga-template-library.md#windows-plugins)
+using Windows, you will need to change the services accordingly. See the [ITL reference](10-icinga-template-library.md#windows-plugins)
for further information.
## <a id="hosts-services"></a> Hosts and Services
The `address` attribute is used by check commands to determine which network
address is associated with the host object.
-Details on troubleshooting check problems can be found [here](16-troubleshooting.md#troubleshooting).
+Details on troubleshooting check problems can be found [here](15-troubleshooting.md#troubleshooting).
### <a id="host-states"></a> Host States
echo requests to the IP address specified in the `address` attribute to determine
whether a host is online.
-A number of other [built-in check commands](7-icinga-template-library.md#plugin-check-commands) are also
+A number of other [built-in check commands](10-icinga-template-library.md#plugin-check-commands) are also
available. In addition to these commands the next few chapters will explain in
detail how to set up your own check commands.
Valid values for custom attributes include:
-* [Strings](18-language-reference.md#string-literals), [numbers](18-language-reference.md#numeric-literals) and [booleans](18-language-reference.md#boolean-literals)
-* [Arrays](18-language-reference.md#array) and [dictionaries](18-language-reference.md#dictionary)
+* [Strings](17-language-reference.md#string-literals), [numbers](17-language-reference.md#numeric-literals) and [booleans](17-language-reference.md#boolean-literals)
+* [Arrays](17-language-reference.md#array) and [dictionaries](17-language-reference.md#dictionary)
* [Functions](3-monitoring-basics.md#custom-attributes-functions)
### <a id="custom-attributes-functions"></a> Functions as Custom Attributes
-Icinga 2 lets you specify [functions](18-language-reference.md#functions) for custom attributes.
+Icinga 2 lets you specify [functions](17-language-reference.md#functions) for custom attributes.
The special case here is that whenever Icinga 2 needs the value for such a custom attribute it runs
the function and uses whatever value the function returns:
vars.text = {{ Math.random() * 100 }}
}
-This example uses the [abbreviated lambda syntax](18-language-reference.md#nullary-lambdas).
+This example uses the [abbreviated lambda syntax](17-language-reference.md#nullary-lambdas).
These functions have access to a number of variables:
}
Acessing object attributes at runtime inside these functions is described in the
-[advanced topics](5-advanced-topics.md#access-object-attributes-at-runtime) chapter.
+[advanced topics](8-advanced-topics.md#access-object-attributes-at-runtime) chapter.
## <a id="runtime-macros"></a> Runtime Macros
## <a id="using-apply"></a> Apply Rules
-Instead of assigning each object ([Service](6-object-types.md#objecttype-service),
-[Notification](6-object-types.md#objecttype-notification), [Dependency](6-object-types.md#objecttype-dependency),
-[ScheduledDowntime](6-object-types.md#objecttype-scheduleddowntime))
-based on attribute identifiers for example `host_name` objects can be [applied](18-language-reference.md#apply).
+Instead of assigning each object ([Service](9-object-types.md#objecttype-service),
+[Notification](9-object-types.md#objecttype-notification), [Dependency](9-object-types.md#objecttype-dependency),
+[ScheduledDowntime](9-object-types.md#objecttype-scheduleddowntime))
+based on attribute identifiers for example `host_name` objects can be [applied](17-language-reference.md#apply).
Before you start using the apply rules keep the following in mind:
* Define the best match.
* A set of unique [custom attributes](3-monitoring-basics.md#custom-attributes) for these hosts/services?
* Or [group](3-monitoring-basics.md#groups) memberships, e.g. a host being a member of a hostgroup, applying services to it?
- * A generic pattern [match](18-language-reference.md#function-calls) on the host/service name?
- * [Multiple expressions combined](3-monitoring-basics.md#using-apply-expressions) with `&&` or `||` [operators](18-language-reference.md#expression-operators)
+ * A generic pattern [match](17-language-reference.md#function-calls) on the host/service name?
+ * [Multiple expressions combined](3-monitoring-basics.md#using-apply-expressions) with `&&` or `||` [operators](17-language-reference.md#expression-operators)
* All expressions must return a boolean value (an empty string is equal to `false` e.g.)
> **Note**
> **Tip**
>
> Building configuration in that dynamic way requires detailed information
-> of the generated objects. Use the `object list` [CLI command](8-cli-commands.md#cli-command-object)
-> after successful [configuration validation](8-cli-commands.md#config-validation).
+> of the generated objects. Use the `object list` [CLI command](11-cli-commands.md#cli-command-object)
+> after successful [configuration validation](11-cli-commands.md#config-validation).
### <a id="using-apply-expressions"></a> Apply Rules Expressions
`OR` the host custom attribute `always_notify` is set to `true`.
The notification is ignored for services whose host name ends with `*internal`
-`OR` the `priority` custom attribute is [less than](18-language-reference.md#expression-operators) `2`.
+`OR` the `priority` custom attribute is [less than](17-language-reference.md#expression-operators) `2`.
template Notification "cust-xy-notification" {
users = [ "noc-xy", "mgmt-xy" ]
ignore where match("*internal", host.name) || (service.vars.priority < 2 && host.vars.is_clustered == true)
}
-More advanced examples are covered [here](5-advanced-topics.md#use-functions-assign-where).
+More advanced examples are covered [here](8-advanced-topics.md#use-functions-assign-where).
### <a id="using-apply-services"></a> Apply Services to Hosts
and all members of the user group `noc` will get notified.
It is also possible to generally apply a notification template and dynamically overwrite values from
-the template by checking for custom attributes. This can be achieved by using [conditional statements](18-language-reference.md#conditional-statements):
+the template by checking for custom attributes. This can be achieved by using [conditional statements](17-language-reference.md#conditional-statements):
apply Notification "host-mail-noc" to Host {
import "mail-host-notification"
The sample configuration includes an example in [downtimes.conf](4-configuring-icinga-2.md#downtimes-conf).
-Detailed examples can be found in the [recurring downtimes](5-advanced-topics.md#recurring-downtimes) chapter.
+Detailed examples can be found in the [recurring downtimes](8-advanced-topics.md#recurring-downtimes) chapter.
### <a id="using-apply-for"></a> Using Apply For Rules
Next to the standard way of using [apply rules](3-monitoring-basics.md#using-apply)
there is the requirement of applying objects based on a set (array or
-dictionary) using [apply for](18-language-reference.md#apply-for) expressions.
+dictionary) using [apply for](17-language-reference.md#apply-for) expressions.
The sample configuration already includes a detailed example in [hosts.conf](4-configuring-icinga-2.md#hosts-conf)
and [services.conf](4-configuring-icinga-2.md#services-conf) for this use case.
Now we want to create service checks for `if01` and `temp`, but not `bgp`.
Furthermore we want to pass the snmp oid stored as dictionary value to the
custom attribute called `vars.snmp_oid` -- this is the command argument required
-by the [snmp](7-icinga-template-library.md#plugin-check-command-snmp) check command.
+by the [snmp](10-icinga-template-library.md#plugin-check-command-snmp) check command.
The service's `display_name` should be set to the identifier inside the dictionary.
apply Service for (identifier => oid in host.vars.oids) {
provided host attributes. For strings, you can use string concatention with the `+` operator.
You can also specifiy the display_name, check command, interval, notes, notes_url, action_url, etc.
-attributes that way. Attribute strings can be [concatenated](18-language-reference.md#expression-operators),
+attributes that way. Attribute strings can be [concatenated](17-language-reference.md#expression-operators),
for example for adding a more detailed service `display_name`.
-This example also uses [if conditions](18-language-reference.md#conditional-statements)
+This example also uses [if conditions](17-language-reference.md#conditional-statements)
if specific values are not set, adding a local default value.
The other way around you can override specific custom attributes inherited from a service template if set.
This example makes use of the [check_iftraffic](https://exchange.icinga.org/exchange/iftraffic) plugin.
The `CheckCommand` definition can be found in the
-[contributed plugin check commands](7-icinga-template-library.md#plugins-contrib-command-iftraffic)
+[contributed plugin check commands](10-icinga-template-library.md#plugin-contrib-command-iftraffic)
-- make sure to include them in your [icinga2 configuration file](4-configuring-icinga-2.md#icinga2-conf).
> **Tip**
>
> Building configuration in that dynamic way requires detailed information
-> of the generated objects. Use the `object list` [CLI command](8-cli-commands.md#cli-command-object)
-> after successful [configuration validation](8-cli-commands.md#config-validation).
+> of the generated objects. Use the `object list` [CLI command](11-cli-commands.md#cli-command-object)
+> after successful [configuration validation](11-cli-commands.md#config-validation).
Verify that the apply-for-rule successfully created the service objects with the
inherited custom attributes:
group.
Details on the `assign where` syntax can be found in the
-[Language Reference](18-language-reference.md#apply).
+[Language Reference](17-language-reference.md#apply).
## <a id="notifications"></a> Notifications
There are many ways of sending notifications, e.g. by email, XMPP,
IRC, Twitter, etc. On its own Icinga 2 does not know how to send notifications.
Instead it relies on external mechanisms such as shell scripts to notify users.
-More notification methods are listed in the [addons and plugins](14-addons-plugins.md#notification-scripts-interfaces)
+More notification methods are listed in the [addons and plugins](13-addons.md#notification-scripts-interfaces)
chapter.
A notification specification requires one or more users (and/or user groups)
If you don't set the `states` and `types` configuration attributes for the `User`
object, notifications for all states and types will be sent.
-Details on troubleshooting notification problems can be found [here](16-troubleshooting.md#troubleshooting).
+Details on troubleshooting notification problems can be found [here](15-troubleshooting.md#troubleshooting).
> **Note**
>
-> Make sure that the [notification](8-cli-commands.md#enable-features) feature is enabled
+> Make sure that the [notification](11-cli-commands.md#enable-features) feature is enabled
> in order to execute notification commands.
You should choose which information you (and your notified users) are interested in
### <a id="check-commands"></a> Check Commands
-[CheckCommand](6-object-types.md#objecttype-checkcommand) objects define the command line how
+[CheckCommand](9-object-types.md#objecttype-checkcommand) objects define the command line how
a check is called.
-[CheckCommand](6-object-types.md#objecttype-checkcommand) objects are referenced by
-[Host](6-object-types.md#objecttype-host) and [Service](6-object-types.md#objecttype-service) objects
+[CheckCommand](9-object-types.md#objecttype-checkcommand) objects are referenced by
+[Host](9-object-types.md#objecttype-host) and [Service](9-object-types.md#objecttype-service) objects
using the `check_command` attribute.
> **Note**
>
-> Make sure that the [checker](8-cli-commands.md#enable-features) feature is enabled in order to
+> Make sure that the [checker](11-cli-commands.md#enable-features) feature is enabled in order to
> execute checks.
#### <a id="command-plugin-integration"></a> Integrate the Plugin with a CheckCommand Definition
-[CheckCommand](6-object-types.md#objecttype-checkcommand) objects require the [ITL template](7-icinga-template-library.md#itl-plugin-check-command)
+[CheckCommand](9-object-types.md#objecttype-checkcommand) objects require the [ITL template](10-icinga-template-library.md#itl-plugin-check-command)
`plugin-check-command` to support native plugin based check methods.
Unless you have done so already, download your check plugin and put it
[-C ca-cert] [-D ca-dir] [-L ciphers] [-f optfile] [-g group]
Next step is to understand how [command parameters](3-monitoring-basics.md#command-passing-parameters)
-are being passed from a host or service object, and add a [CheckCommand](6-object-types.md#objecttype-checkcommand)
+are being passed from a host or service object, and add a [CheckCommand](9-object-types.md#objecttype-checkcommand)
definition based on these required parameters and/or default values.
-Please continue reading in the [plugins section](14-addons-plugins.md#plugins) for additional integration examples.
+Please continue reading in the [plugins section](5-service-monitoring.md#service-monitoring-plugins) for additional integration examples.
#### <a id="command-passing-parameters"></a> Passing Check Command Parameters from Host or Service
by the executed check command.
The check command parameters for ITL provided plugin check command definitions are documented
-[here](7-icinga-template-library.md#plugin-check-commands), for example
-[disk](7-icinga-template-library.md#plugin-check-command-disk).
+[here](10-icinga-template-library.md#plugin-check-commands), for example
+[disk](10-icinga-template-library.md#plugin-check-command-disk).
In order to practice passing command parameters you should [integrate your own plugin](3-monitoring-basics.md#command-plugin-integration).
without SSL enabled checks saving you duplicated command definitions.
Details on all available options can be found in the
-[CheckCommand object definition](6-object-types.md#objecttype-checkcommand).
+[CheckCommand object definition](9-object-types.md#objecttype-checkcommand).
#### <a id="command-environment-variables"></a> Environment Variables
### <a id="notification-commands"></a> Notification Commands
-[NotificationCommand](6-object-types.md#objecttype-notificationcommand) objects define how notifications are delivered to external
+[NotificationCommand](9-object-types.md#objecttype-notificationcommand) objects define how notifications are delivered to external
interfaces (email, XMPP, IRC, Twitter, etc.).
-[NotificationCommand](6-object-types.md#objecttype-notificationcommand) objects are referenced by
-[Notification](6-object-types.md#objecttype-notification) objects using the `command` attribute.
+[NotificationCommand](9-object-types.md#objecttype-notificationcommand) objects are referenced by
+[Notification](9-object-types.md#objecttype-notification) objects using the `command` attribute.
-`NotificationCommand` objects require the [ITL template](7-icinga-template-library.md#itl-plugin-notification-command)
+`NotificationCommand` objects require the [ITL template](10-icinga-template-library.md#itl-plugin-notification-command)
`plugin-notification-command` to support native plugin-based notifications.
> **Note**
>
-> Make sure that the [notification](8-cli-commands.md#enable-features) feature is enabled
+> Make sure that the [notification](11-cli-commands.md#enable-features) feature is enabled
> in order to execute notification commands.
Below is an example using runtime macros from Icinga 2 (such as `$service.output$` for
* The host/service state changes into a [hard state](3-monitoring-basics.md#hard-soft-states)
* The host/service state recovers from a [soft or hard state](3-monitoring-basics.md#hard-soft-states) to [OK](3-monitoring-basics.md#service-states)/[Up](3-monitoring-basics.md#host-states)
-[EventCommand](6-object-types.md#objecttype-eventcommand) objects are referenced by
-[Host](6-object-types.md#objecttype-host) and [Service](6-object-types.md#objecttype-service) objects
+[EventCommand](9-object-types.md#objecttype-eventcommand) objects are referenced by
+[Host](9-object-types.md#objecttype-host) and [Service](9-object-types.md#objecttype-service) objects
using the `event_command` attribute.
Therefore the `EventCommand` object should define a command line
and `$service.state$` will be processed by Icinga 2 helping on fine-granular
events being triggered.
-If you are using a client as [command execution bridge](11-icinga2-client.md#icinga2-client-configuration-command-bridge)
+If you are using a client as [command endpoint](6-distributed-monitoring.md#distributed-monitoring-top-down-command-endpoint)
the event command will be executed on the client itself (similar to the check
command).
icinga ALL=(ALL) NOPASSWD: /etc/init.d/apache2 restart
-Define a generic [EventCommand](6-object-types.md#objecttype-eventcommand) object `event_by_ssh`
+Define a generic [EventCommand](9-object-types.md#objecttype-eventcommand) object `event_by_ssh`
which can be used for all event commands triggered using ssh:
/* pass event commands through ssh */
## <a id="dependencies"></a> Dependencies
-Icinga 2 uses host and service [Dependency](6-object-types.md#objecttype-dependency) objects
+Icinga 2 uses host and service [Dependency](9-object-types.md#objecttype-dependency) objects
for determing their network reachability.
A service can depend on a host, and vice versa. A service has an implicit
dependency will fail and render all child objects (hosts or services) unreachable.
You can determine the child's reachability by querying the `is_reachable` attribute
-in for example [DB IDO](24-appendix.md#schema-db-ido-extensions).
+in for example [DB IDO](23-appendix.md#schema-db-ido-extensions).
### <a id="dependencies-implicit-host-service"></a> Implicit Dependencies for Services on Host
are a good way to start with Icinga 2.
If you're interested in a detailed explanation of each language feature used in those
-configuration files, you can find more information in the [Language Reference](18-language-reference.md#language-reference)
+configuration files, you can find more information in the [Language Reference](17-language-reference.md#language-reference)
chapter.
## <a id="configuration-best-practice"></a> Configuration Best Practice
There are many ways of creating Icinga 2 configuration objects:
* Manually with your preferred editor, for example vi(m), nano, notepad, etc.
-* Generated by a [configuration management tool](14-addons-plugins.md#configuration-tools) such as Puppet, Chef, Ansible, etc.
-* A configuration addon for Icinga 2
+* Generated by a [configuration management tool](13-addons.md#configuration-tools) such as Puppet, Chef, Ansible, etc.
+* A configuration addon for Icinga 2 ([Icinga Director](https://github.com/Icinga/icingaweb2-module-director))
* A custom exporter script from your CMDB or inventory tool
* your own.
* Every plugin used as check, notification or event command requires a `Command` definition.
Further details can be looked up in the [check commands](3-monitoring-basics.md#check-commands) chapter.
+If you are planning to use a distributed monitoring setup with master, satellite and client installations
+take the configuration location into account too. Everything configured on the master, synced to all other
+nodes? Or any specific local configuration (e.g. health checks)?
+
+TODO
+
If you happen to have further questions, do not hesitate to join the
[community support channels](https://support.icinga.org)
and ask community members for their experience and best practices.
* to the documentation that is distributed as part of Icinga 2.
*/
-Icinga 2 supports [C/C++-style comments](18-language-reference.md#comments).
+Icinga 2 supports [C/C++-style comments](17-language-reference.md#comments).
/**
* The constants.conf defines global constants.
This `include` directive takes care of including the configuration files for all
the features which have been enabled with `icinga2 feature enable`. See
-[Enabling/Disabling Features](8-cli-commands.md#enable-features) for more details.
+[Enabling/Disabling Features](11-cli-commands.md#enable-features) for more details.
/**
* The repository.d directory contains all configuration objects
This `include_recursive` directive is used for discovery of services on remote clients
and their generated configuration described in
-[this chapter](11-icinga2-client.md#icinga2-remote-monitoring-master-discovery).
+[this chapter](6-distributed-monitoring.md#distributed-monitoring-bottom-up).
/**
* The `PluginDir` constant must be set to the path where the [Monitoring Project plugins](2-getting-started.md#setting-up-check-plugins) are installed.
This constant is used by a number of
-[built-in check command definitions](7-icinga-template-library.md#plugin-check-commands).
+[built-in check command definitions](10-icinga-template-library.md#plugin-check-commands).
* The `NodeName` constant defines your local node name. Should be set to FQDN which is the default
if not set. This constant is required for local host configuration, monitoring remote clients and
cluster setup.
require a different check command, you can override it in the object definition.
The `vars` attribute can be used to define custom attributes which are available
-for check and notification commands. Most of the [Plugin Check Commands](7-icinga-template-library.md#plugin-check-commands)
+for check and notification commands. Most of the [Plugin Check Commands](10-icinga-template-library.md#plugin-check-commands)
in the Icinga Template Library require an `address` attribute.
The custom attribute `os` is evaluated by the `linux-servers` group in
> **Tip**
>
-> If you don't understand all the attributes and how to use [apply rules](18-language-reference.md#apply),
+> If you don't understand all the attributes and how to use [apply rules](17-language-reference.md#apply),
> don't worry -- the [monitoring basics](3-monitoring-basics.md#monitoring-basics) chapter will explain
> that in detail.
#### <a id="services-conf"></a> services.conf
-These service [apply rules](18-language-reference.md#apply) will show you how to monitor
+These service [apply rules](17-language-reference.md#apply) will show you how to monitor
the local host, but also allow you to re-use or modify them for
your own requirements.
The Debian packages also include an additional `apt` service check applied to the local host.
The command object `icinga` for the embedded health check is provided by the
-[Icinga Template Library (ITL)](7-icinga-template-library.md#icinga-template-library) while `http_ip`, `ssh`, `load`, `processes`,
-`users` and `disk` are all provided by the [Plugin Check Commands](7-icinga-template-library.md#plugin-check-commands)
+[Icinga Template Library (ITL)](10-icinga-template-library.md#icinga-template-library) while `http_ip`, `ssh`, `load`, `processes`,
+`users` and `disk` are all provided by the [Plugin Check Commands](10-icinga-template-library.md#plugin-check-commands)
which we enabled earlier by including the `itl` and `plugins` configuration file.
In this example the `assign where` condition is a boolean expression which is
evaluated for all objects of type `Host` and a new service with name "load"
-is created for each matching host. [Expression operators](18-language-reference.md#expression-operators)
+is created for each matching host. [Expression operators](17-language-reference.md#expression-operators)
may be used in `assign where` conditions.
Multiple `assign where` condition can be combined with `AND` using the `&&` operator
attribute defined `AND` having the custom attribute `os` set to the string
`Linux`.
You can modify this condition to match multiple expressions by combinding `AND`
-and `OR` using `&&` and `||` [operators](18-language-reference.md#expression-operators), for example
+and `OR` using `&&` and `||` [operators](17-language-reference.md#expression-operators), for example
`assign where host.address && (vars.os == "Linux" || vars.os == "Windows")`.
* loop through all entries in the `host.vars.disks` dictionary. That's `disk` and `disk /` as keys.
* call `apply` on each, and set the service object name from the provided key
* inside apply, the `generic-service` template is imported
-* defining the [disk](7-icinga-template-library.md#plugin-check-command-disk) check command requiring command arguments like `disk_partition`
+* defining the [disk](10-icinga-template-library.md#plugin-check-command-disk) check command requiring command arguments like `disk_partition`
* adding the `config` dictionary items to `vars`. Simply said, there's now `vars.disk_partition` defined for the
generated service
defining whether these notifications are applies to hosts or services.
The `import` keyword imports the specific mail templates defined in [templates.conf](4-configuring-icinga-2.md#templates-conf).
-The `interval` attribute is not explicitly set -- it [defaults to 30 minutes](6-object-types.md#objecttype-notification).
+The `interval` attribute is not explicitly set -- it [defaults to 30 minutes](9-object-types.md#objecttype-notification).
By setting the `user_groups` to the value provided by the
respective [host.vars.notification.mail](4-configuring-icinga-2.md#hosts-conf) attribute we'll
custom attribute `os` set to `Linux` and is therefore automatically
a member of the host group `linux-servers`.
-This is done by using the [group assign](18-language-reference.md#group-assign) expressions similar
+This is done by using the [group assign](17-language-reference.md#group-assign) expressions similar
to previously seen [apply rules](3-monitoring-basics.md#using-apply).
object HostGroup "linux-servers" {
}
Service groups can be grouped together by similar pattern matches.
-The [match() function](18-language-reference.md#function-calls) expects a wildcard match string
+The [match() function](17-language-reference.md#function-calls) expects a wildcard match string
and the attribute string to match with.
object ServiceGroup "ping" {
}
The `hostalive` check command is part of the
-[Plugin Check Commands](7-icinga-template-library.md#plugin-check-commands).
+[Plugin Check Commands](10-icinga-template-library.md#plugin-check-commands).
template Notification "mail-host-notification" {
period = "24x7"
}
-More details on `Notification` object attributes can be found [here](6-object-types.md#objecttype-notification).
+More details on `Notification` object attributes can be found [here](9-object-types.md#objecttype-notification).
#### <a id="downtimes-conf"></a> downtimes.conf
The `load` service apply rule defined in [services.conf](4-configuring-icinga-2.md#services-conf) defines
the `backup_downtime` custom attribute.
-The [ScheduledDowntime](6-object-types.md#objecttype-scheduleddowntime) apply rule uses this attribute
+The [ScheduledDowntime](9-object-types.md#objecttype-scheduleddowntime) apply rule uses this attribute
to define the default value for the time ranges required for recurring downtime slots.
apply ScheduledDowntime "backup-downtime" to Service {
#### <a id="satellite-conf"></a> satellite.conf
Includes default templates and dependencies for
-[monitoring remote clients](11-icinga2-client.md#icinga2-client)
+[monitoring remote clients](6-distributed-monitoring.md#distributed-monitoring)
using service discovery and
-[config generation](11-icinga2-client.md#icinga2-remote-monitoring-master-discovery)
+[config generation](6-distributed-monitoring.md#distributed-monitoring-bottom-up)
on the master. Can be ignored/removed on setups not using this feature.
#### <a id="api-users-conf"></a> api-users.conf
-Provides the default [ApiUser](6-object-types.md#objecttype-apiuser) object
-named "root" for the [API authentication](9-icinga2-api.md#icinga2-api-authentication).
+Provides the default [ApiUser](9-object-types.md#objecttype-apiuser) object
+named "root" for the [API authentication](12-icinga2-api.md#icinga2-api-authentication).
#### <a id="app-conf"></a> app.conf
-Provides the default [IcingaApplication](6-object-types.md#objecttype-icingaapplication)
+Provides the default [IcingaApplication](9-object-types.md#objecttype-icingaapplication)
object named "app" for additional settings such as disabling notifications
globally, etc.
--- /dev/null
+# <a id="service-monitoring"></a> Service Monitoring
+
+The power of Icinga 2 lies in its modularity. There are thousands of
+community plugins available next to the standard plugins provided by
+the [Monitoring Plugins project](https://www.monitoring-plugins.org).
+
+## <a id="service-monitoring-requirements"></a> Requirements
+
+### <a id="service-monitoring-plugins"></a> Plugins
+
+All existing Nagios or Icinga 1.x plugins work with Icinga 2. Here's a
+list of popular community sites which host check plugins:
+
+* [Icinga Exchange](https://exchange.icinga.org)
+* [Icinga Wiki](https://wiki.icinga.org)
+
+The recommended way of setting up these plugins is to copy them to a common directory
+and create a new global constant, e.g. `CustomPluginDir` in your [constants.conf](4-configuring-icinga-2.md#constants-conf)
+configuration file:
+
+ # cp check_snmp_int.pl /opt/monitoring/plugins
+ # chmod +x /opt/plugins/check_snmp_int.pl
+
+ # cat /etc/icinga2/constants.conf
+ /**
+ * This file defines global constants which can be used in
+ * the other configuration files. At a minimum the
+ * PluginDir constant should be defined.
+ */
+
+ const PluginDir = "/usr/lib/nagios/plugins"
+ const CustomPluginDir = "/opt/monitoring/plugins"
+
+Prior to using the check plugin with Icinga 2 you should ensure that it is working properly
+by trying to run it on the console using whichever user Icinga 2 is running as:
+
+ # su - icinga -s /bin/bash
+ $ /opt/monitoring/plugins/check_snmp_int.pl --help
+
+Additional libraries may be required for some plugins. Please consult the plugin
+documentation and/or plugin provided README for installation instructions.
+Sometimes plugins contain hard-coded paths to other components. Instead of changing
+the plugin it might be easier to create logical links which is (more) update-safe.
+
+Sometimes there are plugins which do not exactly fit your requirements.
+Either you'll modify an existing plugin or you'll write one by yourself.
+
+### <a id="service-monitoring-plugin-checkcommand"></a> CheckCommand Definition
+
+Each plugin requires a [CheckCommand](9-object-types.md#objecttype-checkcommand) object in your
+configuration which can be used in the [Service](9-object-types.md#objecttype-service) or
+[Host](9-object-types.md#objecttype-host) object definition.
+
+Please check if the Icinga 2 package already provides an
+[existing CheckCommand definition](10-icinga-template-library.md#plugin-check-commands).
+If yes throroughly check the required parameters and integrate the check command
+into your host and service objects.
+
+There are the following conventions to follow when adding a new command object definition:
+
+* Always import the `plugin-check-command` template
+* Use [command-arguments](#) whenever possible. The `command` attribute must be an array
+in `[ ... ]` then for shell escaping.
+* Define a unique `prefix` for the command's specific command arguments. That way you can safely
+set them on host/service level and you'll always know which command they control.
+* Use command argument default values, e.g. for thresholds
+* Use [advanced conditions](9-object-types.md#objecttype-checkcommand) like `set_if` definitions.
+
+Example for a custom `my-snmp-int` check command:
+
+ object CheckCommand "my-snmp-int" {
+ import "plugin-check-command"
+
+ command = [ CustomPluginDir + "/check_snmp_int.pl" ]
+
+ arguments = {
+ "-H" = "$snmp_address$"
+ "-C" = "$snmp_community$"
+ "-p" = "$snmp_port$"
+ "-2" = {
+ set_if = "$snmp_v2$"
+ }
+ "-n" = "$snmp_interface$"
+ "-f" = {
+ set_if = "$snmp_perf$"
+ }
+ "-w" = "$snmp_warn$"
+ "-c" = "$snmp_crit$"
+ }
+
+ vars.snmp_v2 = true
+ vars.snmp_perf = true
+ vars.snmp_warn = "300,400"
+ vars.snmp_crit = "0,600"
+ }
+
+
+For further information on your monitoring configuration read the
+[Monitoring Basics](3-monitoring-basics.md#monitoring-basics) chapter.
+
+If you have created your own `CheckCommand` definition please kindly
+[send it upstream](https://wiki.icinga.org/display/community/Contribute+Icinga+2+ITL+Plugin+Check+Command+Definitions)
+
+### <a id="service-monitoring-plugin-api"></a> Plugin API
+
+Currently Icinga 2 supports the native plugin API specification from the `Monitoring Plugins`
+project.
+
+The `Monitoring Plugin API` is defined in the [Monitoring Plugins Development Guidelines](https://www.monitoring-plugins.org/doc/guidelines.html).
+
+## <a id="service-monitoring-overview"></a> Service Monitoring Overview
+
+The following examples should get you started with your own integration ideas.
+There is a variety of common plugins available. This collection is not complete --
+if you have any updates please send a documentation patch upstream.
+
+## <a id="service-monitoring-general"></a> General Monitoring
+
+If the remote service is available using a network protocol and port,
+and a check plugin is available, you don't necessarily need a local client installed.
+Rather choose a plugin and configure all parameters and thresholds. The [Icinga 2 Template Library](10-icinga-template-library.md#icinga-template-library)
+already ships various examples like
+
+* [ping4](10-icinga-template-library.md#plugin-check-command-ping4), [ping6](10-icinga-template-library.md#plugin-check-command-ping6),
+[fping4](10-icinga-template-library.md#plugin-check-command-fping4), [fping6](10-icinga-template-library.md#plugin-check-command-fping6), [hostalive](10-icinga-template-library.md#plugin-check-command-hostalive)
+* [tcp](10-icinga-template-library.md#plugin-check-command-tcp), [udp](10-icinga-template-library.md#plugin-check-command-udp), [ssl](10-icinga-template-library.md#plugin-check-command-ssl)
+* [ntp_time](10-icinga-template-library.md#plugin-check-command-ntp-time)
+
+## <a id="service-monitoring-linux"></a> Linux Monitoring
+
+* [disk](10-icinga-template-library.md#plugin-check-command-disk)
+* [mem](10-icinga-template-library.md#plugin-contrib-command-mem), [swap](10-icinga-template-library.md#plugin-check-command-swap)
+* [running_kernel](10-icinga-template-library.md#plugin-contrib-command-running_kernel)
+* Package repositores ([apt](10-icinga-template-library.md#plugin-check-command-apt), [yum](10-icinga-template-library.md#plugin-contrib-command-yum), etc.)
+* [ssh](10-icinga-template-library.md#plugin-check-command-ssh)
+* Performance ([iostat](10-icinga-template-library.md#plugin-contrib-command-iostat), [check_sar_perf](https://github.com/dnsmichi/icinga-plugins/blob/master/scripts/check_sar_perf.py))
+
+## <a id="service-monitoring-windows"></a> Windows Monitoring
+
+* [check_wmi_plus](http://www.edcint.co.nz/checkwmiplus/)
+* [NSClient++](https://www.nsclient.org) (in combination with the Icinga 2 client as [nscp-local](10-icinga-template-library.md#nscp-plugin-check-commands) check commands)
+* [Icinga 2 Windows Plugins](10-icinga-template-library.md#windows-plugins) (disk, load, memory, network, performance counters, ping, procs, service, swap, updates, uptime, users
+* vbs and Powershell scripts
+
+## <a id="service-monitoring-database"></a> Database Monitoring
+
+* MySQL ([mysql_health](10-icinga-template-library.md#plugin-contrib-command-mysql_health), [mysql](10-icinga-template-library.md#plugin-check-command-mysql), [mysql_query](10-icinga-template-library.md#plugin-check-command-mysql-query))
+* PostgreSQL ([postgres](10-icinga-template-library.md#plugin-contrib-command-postgres))
+* Oracle ([oracle_health](10-icinga-template-library.md#plugin-contrib-command-oracle_health))
+* MSSQL ([mssql_health](10-icinga-template-library.md#plugin-contrib-command-mssql_health))
+* DB2 ([db2_health](10-icinga-template-library.md#plugin-contrib-command-db2_health))
+* MongoDB ([db2_health](10-icinga-template-library.md#plugin-contrib-command-mongodb))
+* Elasticsearch ([db2_health](10-icinga-template-library.md#plugin-contrib-command-elasticsearch))
+* Redis ([db2_health](10-icinga-template-library.md#plugin-contrib-command-redis))
+
+## <a id="service-monitoring-snmp"></a> SNMP Monitoring
+
+* [Manubulon plugins](10-icinga-template-library.md#snmp-manubulon-plugin-check-commands) (interface, storage, load, memory, process)
+* [snmp](10-icinga-template-library.md#plugin-check-command-snmp), [snmpv3](10-icinga-template-library.md#plugin-check-command-snmpv3)
+
+## <a id="service-monitoring-network"></a> Network Monitoring
+
+* [nwc_health](10-icinga-template-library.md#plugin-contrib-command-nwc_health)
+* [interfaces](10-icinga-template-library.md#plugin-contrib-command-interfaces)
+* [interfacetable](10-icinga-template-library.md#plugin-contrib-command-interfacetable)
+* [iftraffic](10-icinga-template-library.md#plugin-contrib-command-iftraffic), [iftraffic64](10-icinga-template-library.md#plugin-contrib-command-iftraffic64)
+
+## <a id="service-monitoring-web"></a> Web Monitoring
+
+* [http](10-icinga-template-library.md#plugin-check-command-http)
+* [ftp](10-icinga-template-library.md#plugin-check-command-ftp)
+* [webinject](10-icinga-template-library.md#plugin-contrib-command-webinject)
+* [squid](10-icinga-template-library.md#plugin-contrib-command-squid)
+* [apache_status](10-icinga-template-library.md#plugin-contrib-command-apache_status)
+* [nginx_status](10-icinga-template-library.md#plugin-contrib-command-nginx_status)
+* [kdc](10-icinga-template-library.md#plugin-contrib-command-kdc)
+* [rbl](10-icinga-template-library.md#plugin-contrib-command-rbl)
+
+## <a id="service-monitoring-java"></a> Java Monitoring
+
+* [jmx4perl](10-icinga-template-library.md#plugin-contrib-command-jmx4perl)
+
+## <a id="service-monitoring-dns"></a> DNS Monitoring
+
+* [dns](10-icinga-template-library.md#plugin-check-command-dns)
+* [dig](10-icinga-template-library.md#plugin-check-command-dig)
+* [dhcp](10-icinga-template-library.md#plugin-check-command-dhcp)
+
+## <a id="service-monitoring-backup"></a> Backup Monitoring
+
+* [check_bareos](https://github.com/widhalmt/check_bareos)
+
+## <a id="service-monitoring-log"></a> Log Monitoring
+
+* [check_logfiles](https://labs.consol.de/nagios/check_logfiles/)
+* [check_logstash](https://github.com/widhalmt/check_logstash)
+* [check_graylog2_stream](https://github.com/Graylog2/check-graylog2-stream)
+
+## <a id="service-monitoring-virtualization"></a> Virtualization Monitoring
+
+* [esxi_hardware](10-icinga-template-library.md#plugin-contrib-command-esxi-hardware)
+* [VMWare](10-icinga-template-library.md#plugin-contrib-vmware)
+
+## <a id="service-monitoring-sap"></a> SAP Monitoring
+
+* [check_sap_health](https://labs.consol.de/nagios/check_sap_health/index.html)
+* [SAP CCMS](https://sourceforge.net/projects/nagios-sap-ccms/)
+
+## <a id="service-monitoring-mail"></a> Mail Monitoring
+
+* [smtp](10-icinga-template-library.md#plugin-check-command-smtp), [ssmtp](10-icinga-template-library.md#plugin-check-command-ssmtp)
+* [imap](10-icinga-template-library.md#plugin-check-command-imap), [simap](10-icinga-template-library.md#plugin-check-command-simap)
+* [pop](10-icinga-template-library.md#plugin-check-command-pop), [spop](10-icinga-template-library.md#plugin-check-command-spop)
+
+## <a id="service-monitoring-hardware"></a> Hardware Monitoring
+
+* [hpasm](10-icinga-template-library.md#plugin-contrib-command-hpasm)
+* [ipmi-sensor](10-icinga-template-library.md#plugin-contrib-command-ipmi-sensor)
+
+## <a id="service-monitoring-metrics"></a> Metrics Monitoring
+
+* [graphite](10-icinga-template-library.md#plugin-contrib-command-graphite)
--- /dev/null
+# <a id="distributed-monitoring"></a> Distributed Monitoring with Masters, Satellites and Clients
+
+This chapter will guide you through the setup of a distributed monitoring
+environment. This includes High-Availability clustering and Icinga 2 client
+setup details.
+
+## <a id="distributed-monitoring-roles"></a> Roles: Master, Satellite and Clients
+
+Icinga 2 nodes can be given names for easier understanding:
+
+* A `master` node which is on top of the hierarchy
+* A `satellite` node which is a child of a `master` node
+* A `client` node which works as an `agent` connected to `master` and/or `satellite` nodes
+
+The following sections will refer to these roles and explain
+their possibilities and differences in detail.
+
+> **Tip**:
+>
+> If you just want to install a single master node with several hosts
+> monitored with Icinga 2 clients continue reading, we'll start with
+> simple examples.
+> In case you are planning a huge cluster setup with multiple levels and
+> lots of clients - read on, we'll deal with these examples later on.
+
+The installation on each system is the same -- you need to install the
+Icinga 2 package. The required configuration steps are mostly helped with CLI commands.
+
+The first thing you need learn about a distributed setup -- the overall hierarchy.
+
+## <a id="distributed-monitoring-zones"></a> Zones
+
+The Icinga 2 hierarchy consists of so-called [Zone](9-object-types.md#objecttype-zone) objects.
+Zones depend on a parent-child relationship for trusting each other.
+
+
+
+Example for the satellite zones which have the `master` zone as parent zone:
+
+ object Zone "master" {
+ //...
+ }
+
+ object Zone "satellite region 1" {
+ parent = "master"
+ //...
+ }
+
+ object Zone "satellite region 2" {
+ parent = "master"
+ //...
+ }
+
+There are certain limitations for child zones - e.g. their members are not allowed
+to send configuration to the parent zone members. Vice versa the
+trust hierarchy allows for example the `master` zone to send
+configuration files to the `satellite` zone. Read more about this
+in the [security section](6-distributed-monitoring.md#distributed-monitoring-security).
+
+`client` nodes also have their own unique zone. By convention you
+can use the FQDN for the zone name.
+
+## <a id="distributed-monitoring-endpoints"></a> Endpoints
+
+Nodes which are a member of a zone are so-called [Endpoint](9-object-types.md#objecttype-endpoint) objects.
+
+
+
+Example configuration for two endpoints in different zones:
+
+ object Endpoint "icinga2-master1.localdomain" {
+ host = "192.168.56.101"
+ }
+
+ object Endpoint "icinga2-satellite1.localdomain" {
+ host = "192.168.56.105"
+ }
+
+ object Zone "master" {
+ endpoints = [ "icinga2-master1.localdomain" ]
+ }
+
+ object Zone "satellite" {
+ endpoints = [ "icinga2-satellite1.localdomain" ]
+ parent = "master"
+ }
+
+All endpoints in the same zone work as High-Availability setup. If you have for
+example two nodes in the `master` zone, they will load-balance the check execution.
+
+Endpoint objects are important for specifying the connection
+information e.g. if the master should actively try to connect to a client.
+
+The zone membership is defined inside the `Zone` object definition using
+the `endpoints` attribute with an array of `Endpoint` names.
+
+## <a id="distributed-monitoring-apilistener"></a> ApiListener
+
+In case you are using the CLI commands later you won't directly see
+this configuration object. The [ApiListener](9-object-types.md#objecttype-apilistener)
+object is used to load the SSL certificates and specify restrictions
+for e.g. accepting configuration.
+
+It is further used for the [Icinga 2 REST API](12-icinga2-api.md#icinga2-api) which shares
+the same host and port with the Icinga 2 Cluster protocol.
+
+The object configuration is stored as feature in `/etc/icinga2/features-enabled/api.conf`
+by default.
+
+In order to use the `api` feature you need to enable it and restart Icinga 2.
+
+ icinga2 feature enable api
+
+## <a id="distributed-monitoring-conventions"></a> Conventions
+
+By convention all nodes should be configured using their FQDN.
+
+Furthermore you must ensure that the following configuration names
+are exact the same:
+
+* Host certificate common name (CN)
+* Endpoint configuration object
+* NodeName constant
+
+The cli commands will help you here already and minimize the effort. Just keep in mind
+-- use the FQDN for endpoints and common names when asked.
+
+## <a id="distributed-monitoring-security"></a> Security
+
+While there are certain capabilities to ensure the safe communication between all
+nodes (firewalls, policies, software hardening, etc.) the Icinga 2 also provides
+additional security itself:
+
+* SSL certificates are mandatory for cluster communication. There are CLI commands helping with their creation.
+* Child zones only receive event updates (check results, commands, etc.) for their configured updates.
+* Zones cannot influence/interfere other zones. Each checked object is assigned to only one zone.
+* All nodes in a zone trust each other.
+* [Config sync]() and [remote command endpoint execution]() is disabled by default.
+
+## <a id="distributed-monitoring-setup-master"></a> Master Setup
+
+This section explains how to install a central single master node using
+the `node wizard` command. If you prefer to do a manual installation please
+refer to the [manual setup]() section.
+
+Required information:
+
+ Parameter | Description
+ --------------------|--------------------
+ Common name (CN) | **Required.** By convention this should be the host's FQDN. Defaults to the FQDN.
+ API bind host | **Optional.** Allows to specify the address where the ApiListener is bound to. For advanced usage only.
+ API bind port | **Optional.** Allows to specify the port where the ApiListener is bound to. For advanced usage only (requires changing the default port 5665 everywhere).
+
+The setup wizard will ensure that the following steps are taken:
+
+* Setup the `api` feature
+* Generate a new certificate authority (CA) in `/var/lib/icinga2/ca` if not existing
+* Create a certificate signing request (CSR) for the local node
+* Sign the CSR with the local CA and copy all files into the `/etc/icinga2/pki` directory
+* Update the `zones.conf` file with the new zone hierarchy
+* Update `/etc/icinga2/features-enabled/api.conf` and `constants.conf`
+
+Example master setup for the `icinga2-master1.localdomain` node on CentOS 7:
+
+ [root@icinga2-master1.localdomain /]# icinga2 node wizard
+ Welcome to the Icinga 2 Setup Wizard!
+
+ We'll guide you through all required configuration details.
+
+ Please specify if this is a satellite setup ('n' installs a master setup) [Y/n]: n
+ Starting the Master setup routine...
+ Please specifiy the common name (CN) [icinga2-master1.localdomain]: icinga2-master1.localdomain
+ Checking for existing certificates for common name 'icinga2-master1.localdomain'...
+ Certificates not yet generated. Running 'api setup' now.
+ information/cli: Generating new CA.
+ information/base: Writing private key to '/var/lib/icinga2/ca/ca.key'.
+ information/base: Writing X509 certificate to '/var/lib/icinga2/ca/ca.crt'.
+ information/cli: Generating new CSR in '/etc/icinga2/pki/icinga2-master1.localdomain.csr'.
+ information/base: Writing private key to '/etc/icinga2/pki/icinga2-master1.localdomain.key'.
+ information/base: Writing certificate signing request to '/etc/icinga2/pki/icinga2-master1.localdomain.csr'.
+ information/cli: Signing CSR with CA and writing certificate to '/etc/icinga2/pki/icinga2-master1.localdomain.crt'.
+ information/cli: Copying CA certificate to '/etc/icinga2/pki/ca.crt'.
+ Generating master configuration for Icinga 2.
+ information/cli: Adding new ApiUser 'root' in '/etc/icinga2/conf.d/api-users.conf'.
+ information/cli: Enabling the 'api' feature.
+ Enabling feature api. Make sure to restart Icinga 2 for these changes to take effect.
+ information/cli: Dumping config items to file '/etc/icinga2/zones.conf'.
+ information/cli: Created backup file '/etc/icinga2/zones.conf.orig'.
+ Please specify the API bind host/port (optional):
+ Bind Host []:
+ Bind Port []:
+ information/cli: Created backup file '/etc/icinga2/features-available/api.conf.orig'.
+ information/cli: Updating constants.conf.
+ information/cli: Created backup file '/etc/icinga2/constants.conf.orig'.
+ information/cli: Updating constants file '/etc/icinga2/constants.conf'.
+ information/cli: Updating constants file '/etc/icinga2/constants.conf'.
+ information/cli: Updating constants file '/etc/icinga2/constants.conf'.
+ Done.
+
+ Now restart your Icinga 2 daemon to finish the installation!
+
+ [root@icinga2-master1.localdomain /]# systemctl restart icinga2
+
+The CA public and private key are stored in the `/var/lib/icinga2/ca` directory. Keep this path secure and include
+it in your backups.
+
+Once the master setup is completed, you can also use this node as primary CSR auto-signing
+master. The following chapter will explain how to use the cli commands fetching their
+signed certificate from this master node.
+
+## <a id="distributed-monitoring-setup-satellite-client"></a> Client/Satellite Setup
+
+This section describes the setup of a satellite and/or client connected to an
+existing master node setup. If you haven't done so already please [run the master setup](6-distributed-monitoring.md#distributed-monitoring-setup-master).
+
+Icinga 2 on the master node must be running and accepting connections on port `5665`.
+
+The `node wizard` cli command will setup a satellite/client using CSR auto-signing. This
+involves that the setup wizard sends a certificate signing request (CSR) to the
+master node.
+There is a security mechanism in place which requires the client to send in a valid
+ticket for CSR auto-signing. This ticket must be generated on the master beforehand.
+
+Required information:
+
+ Parameter | Description
+ --------------------|--------------------
+ Common name (CN) | **Required.** The common name for the satellite/client. By convention this should be the FQDN.
+
+Example for the client `icinga2-client1.localdomain` generating a ticket on the master node
+`icinga2-master1.localdomain`:
+
+ [root@icinga2-master1.localdomain /]# icinga2 pki ticket --cn icinga2-client1.localdomain
+ 4f75d2ecd253575fe9180938ebff7cbca262f96e
+
+Store that ticket number for the satellite/client setup below.
+
+### <a id="distributed-monitoring-setup-client-linux"></a> Client/Satellite Linux Setup
+
+Please ensure that you've run all the steps mentioned in the [client/satellite chapter](6-distributed-monitoring.md#distributed-monitoring-setup-satellite-client).
+
+Required information:
+
+ Parameter | Description
+ --------------------|--------------------
+ Common name (CN) | **Required.** By convention this should be the host's FQDN. Defaults to the FQDN.
+ Master common name | **Required.** Use the common name you've specified for your master node before.
+ Establish connection to the master | **Optional.** Whether the client should attempt to connect the to master or not. Defaults to `y`.
+ Master endpoint host | **Required if the the client should connect to the master.** The master's IP address or FQDN. This information is written to the `Endpoint` object configuration in the `zones.conf` file.
+ Master endpoint port | **Optional if the the client should connect to the master.** The master's listening port. This information is written to the `Endpoint` object configuration.
+ Add more master endpoints | **Optional.** If you have multiple master nodes configured, add them here.
+ Master connection for CSR auto-signing | **Required.** The master node's IP address or FQDN and port where the client should request a certificate from. Defaults to the master endpoint host.
+ Certificate information | **Required.** Verify that the connecting host really is the requested master node.
+ Request ticket | **Required.** Paste the previously generated ticket number from the master node here.
+ API bind host | **Optional.** Allows to specify the address where the ApiListener is bound to. For advanced usage only.
+ API bind port | **Optional.** Allows to specify the port where the ApiListener is bound to. For advanced usage only (requires changing the default port 5665 everywhere).
+ Accept config | **Optional.** Whether this node accepts configuration sync from the master node (required for [config sync mode](6-distributed-monitoring.md#distributed-monitoring-top-down-config-sync). Defaults to 'n'.
+ Accept commands | **Optional.** Whether this node accepts command execution message from the master node (required for [command endpoint mode](6-distributed-monitoring.md#distributed-monitoring-top-down-command-endpoint). Defaults to 'n'.
+
+
+Example for the client `icinga2-client1.localdomain` generating a ticket on the master node
+`icinga2-master1.localdomain`:
+
+ [root@icinga2-master1.localdomain /]# icinga2 pki ticket --cn icinga2-client1.localdomain
+ 4f75d2ecd253575fe9180938ebff7cbca262f96e
+
+Example client setup for the `icinga2-client1.localdomain` node on CentOS 7. This client
+is configured to receive configuration sync and also accept commands.
+
+ [root@icinga2-client1.localdomain /]# icinga2 node wizard
+ Welcome to the Icinga 2 Setup Wizard!
+
+ We'll guide you through all required configuration details.
+
+ Please specify if this is a satellite setup ('n' installs a master setup) [Y/n]:
+ Starting the Node setup routine...
+ Please specifiy the common name (CN) [icinga2-client1.localdomain]: icinga2-client1.localdomain
+ Please specify the master endpoint(s) this node should connect to:
+ Master Common Name (CN from your master setup): icinga2-master1.localdomain
+ Do you want to establish a connection to the master from this node? [Y/n]:
+ Please fill out the master connection information:
+ Master endpoint host (Your master's IP address or FQDN): 192.168.56.101
+ Master endpoint port [5665]:
+ Add more master endpoints? [y/N]:
+ Please specify the master connection for CSR auto-signing (defaults to master endpoint host):
+ Host [192.168.56.101]: 192.168.2.101
+ Port [5665]:
+ information/base: Writing private key to '/etc/icinga2/pki/icinga2-client1.localdomain.key'.
+ information/base: Writing X509 certificate to '/etc/icinga2/pki/icinga2-client1.localdomain.crt'.
+ information/cli: Fetching public certificate from master (192.168.56.101, 5665):
+
+ Certificate information:
+
+ Subject: CN = icinga2-master1.localdomain
+ Issuer: CN = Icinga CA
+ Valid From: Feb 23 14:45:32 2016 GMT
+ Valid Until: Feb 19 14:45:32 2031 GMT
+ Fingerprint: AC 99 8B 2B 3D B0 01 00 E5 21 FA 05 2E EC D5 A9 EF 9E AA E3
+
+ Is this information correct? [y/N]: y
+ information/cli: Received trusted master certificate.
+
+ Please specify the request ticket generated on your Icinga 2 master.
+ (Hint: # icinga2 pki ticket --cn 'icinga2-client1.localdomain'): 4f75d2ecd253575fe9180938ebff7cbca262f96e
+ information/cli: Requesting certificate with ticket '4f75d2ecd253575fe9180938ebff7cbca262f96e'.
+
+ information/cli: Created backup file '/etc/icinga2/pki/icinga2-client1.localdomain.crt.orig'.
+ information/cli: Writing signed certificate to file '/etc/icinga2/pki/icinga2-client1.localdomain.crt'.
+ information/cli: Writing CA certificate to file '/etc/icinga2/pki/ca.crt'.
+ Please specify the API bind host/port (optional):
+ Bind Host []:
+ Bind Port []:
+ Accept config from master? [y/N]: y
+ Accept commands from master? [y/N]: y
+ information/cli: Disabling the Notification feature.
+ Disabling feature notification. Make sure to restart Icinga 2 for these changes to take effect.
+ information/cli: Enabling the Apilistener feature.
+ information/cli: Generating local zones.conf.
+ information/cli: Dumping config items to file '/etc/icinga2/zones.conf'.
+ information/cli: Updating constants.conf.
+ information/cli: Updating constants file '/etc/icinga2/constants.conf'.
+ information/cli: Updating constants file '/etc/icinga2/constants.conf'.
+ Done.
+
+ Now restart your Icinga 2 daemon to finish the installation!
+
+Now that you've succesfully installed a satellite/client please proceed to the
+[configuration modes](6-distributed-monitoring.md#distributed-monitoring-configuration-modes).
+
+
+### <a id="distributed-monitoring-setup-client-windows"></a> Client/Satellite Windows Setup
+
+Download the MSI-Installer package from [http://packages.icinga.org/windows/](http://packages.icinga.org/windows/).
+
+Requirements:
+
+* Windows Vista/Server 2008 or higher
+* [Microsoft .NET Framework 2.0](http://www.microsoft.com/de-de/download/details.aspx?id=1639) if not already installed.
+
+The installer package includes the [NSClient++](http://www.nsclient.org/) so Icinga 2 can
+use its built-in plugins. You can use the [nscp-local commands from the ITL](10-icinga-template-library.md#nscp-plugin-check-commands)
+for these plugins.
+
+
+
+
+
+
+
+The graphical installer will offer to run the Icinga 2 setup wizard after the installation.
+You can also manually run the Icinga 2 setup wizard from the start menu.
+
+On a fresh installation the setup wizard will guide you through the initial configuration
+as well as the required details for SSL certificate generation using CSR-Autosigning.
+
+You'll need the following configuration details:
+
+* The client common name (CN). Defaults to FQDN.
+* The request ticket number generated on your master for CSR Auto-Signing
+
+Example for the client `icinga2-client2.localdomain` generating a ticket on the master node
+`icinga2-master1.localdomain`:
+
+ [root@icinga2-master1.localdomain /]# icinga2 pki ticket --cn DESKTOP-IHRPO96
+
+Fill in the required information and click `Add` to add a new master connection.
+
+
+
+Add the following details:
+
+* The master endpoint name. Look into your master setup `zones.conf` file for the proper name.
+* The master endpoint connection information. Your master's IP address and port (defaults to 5665)
+
+
+
+You can optionally enable the following settings:
+
+* Accept config updates from master (client with [config sync mode](6-distributed-monitoring.md#distributed-monitoring-top-down-config-sync))
+* Accept commands from master (client as [command endpoint](6-distributed-monitoring.md#distributed-monitoring-top-down-command-endpoint)).
+* Install/Update NSClient++
+
+
+
+The next step allows you to verify the CA presented by the master.
+
+
+
+If you have chosen to install/update the NSClient++ package, the Icinga 2 setup wizard will ask
+you to do so.
+
+
+
+Finish the setup wizard.
+
+
+
+Once install and configuration is done, Icinga 2 is automatically started as a Windows service.
+
+
+
+The Icinga 2 configuration is located inside the `C:\ProgramData\icinga2` directory.
+If you click `Examine Config` in the setup wizard, it will open a new explorer window.
+
+
+
+The configuration files can be modified with your favorite editor.
+
+Configuration validation is done similar to the linux pendant on the Windows shell:
+
+ C:> icinga2.exe daemon -C
+
+**Note**: You have to run this command in a shell with `administrator` permissions.
+
+In case you want to restart the Icinga 2 service, run `services.msc` and restart the
+`icinga2` service. Alternatively you can use the `net {start,stop}` CLI commands.
+
+Now that you've succesfully installed a satellite/client please proceed to the
+[configuration modes](6-distributed-monitoring.md#distributed-monitoring-configuration-modes).
+
+## <a id="distributed-monitoring-configuration-modes"></a> Configuration Modes
+
+There are different ways to ensure that the Icinga 2 cluster nodes execute
+checks, send notifications, etc.
+
+The following modes differ in the way the host/service object
+configuration is synchronized among nodes and checks are executed.
+
+* [Top down](6-distributed-monitoring.md#distributed-monitoring-top-down). This mode syncs the configuration and commands from the master into child zones.
+* [Bottom up](6-distributed-monitoring.md#distributed-monitoring-bottom-up). This mode leaves the configuration on the child nodes and requires an import on the parent nodes.
+
+Read the chapter carefully and decide upon your requirements which way fits
+best for your environments. You should not mix them -- that will overly complicate
+your setup.
+
+Check results are synced all the way up from the child nodes to the parent nodes.
+That happens automatically and is ensured by the cluster protocol.
+
+### <a id="distributed-monitoring-top-down"></a> Top Down
+
+This is the most commonly used mode gathered from community feedback.
+
+There are two different behaviours with check execution:
+
+* Send a command execution event remotely, the scheduler still runs on the parent node
+* Sync the host/service objects directly to the child node, checks are executed locally
+
+Again -- it does not matter whether this is a `client` or a `satellite`
+which is receiving configuration or command execution events.
+
+### <a id="distributed-monitoring-top-down-command-endpoint"></a> Top Down Command Endpoint
+
+This mode will force the Icinga 2 node to execute commands remotely on a specified endpoint.
+The host/service object configuration is located on the master/satellite and the client only
+needs the CheckCommand object definitions used.
+
+
+
+Advantages:
+
+* No local checks defined on the child node (client)
+* Light-weight remote check execution (asynchronous events)
+* No replay log necessary on child node disconnect (ensure to set `log_duration=0` on the parent node)
+* Pin checks to specific endpoints (if the child zone consists of 2 endpoints)
+
+Disadvantages:
+
+* If the child node is not connected, no more checks are executed
+* Requires additional configuration attribute specified in host/service objects
+* Requires local `CheckCommand` object configuration. Best practice is to use a [global config zone](6-distributed-monitoring.md#distributed-monitoring-global-zone-config-sync).
+
+In order that all involved nodes will accept configuration and/or
+commands you'll need to configure the `Zone` and `Endpoint` hierarchy
+on all nodes.
+
+* `icinga2-master1.localdomain` is the configuration master in this scenario.
+* `icinga2-client2.localdomain` acts as client which receives command execution messages via command endpoint from the master. In addition it receives global check command configuration from the master.
+
+Put the endpoint and zone configuration on **both** nodes into `/etc/icinga2/zones.conf`.
+
+The endpoint configuration could look like this:
+
+ object Endpoint "icinga2-master1.localdomain" {
+ host = "192.168.56.101"
+ }
+
+ object Endpoint "icinga2-client2.localdomain" {
+ host = "192.168.56.112"
+ }
+
+Then you'll need to define two zones. There is no naming convention but best practice
+is to either use `master`, `satellite`/`client-fqdn` or go by region names.
+
+> **Note**
+>
+> Each client requires its own zone and endpoint configuration. Best practice
+> has been to use the client's FQDN for all object names.
+
+The `master` zone is a parent of the `icinga2-client2.localdomain` zone.
+
+ object Zone "master" {
+ endpoints = [ "icinga2-master1.localdomain" ] //array with endpoint names
+ }
+
+ object Zone "icinga2-client2.localdomain" {
+ endpoints = [ "icinga2-client2.localdomain" ]
+
+ parent = "master" //establish zone hierarchy
+ }
+
+In addition to that add a [global zone](6-distributed-monitoring.md#distributed-monitoring-global-zone-config-sync)
+for syncing check commands later.
+
+ object Zone "global-templates" {
+ global = true
+ }
+
+Edit the `api` feature on the client `icinga2-client2.localdomain` in
+the `/etc/icinga2/features-enabled/api.conf` file and ensure to set
+`accept_commands` and `accept_config` to `true`.
+
+ [root@icinga2-client1.localdomain /]# vim /etc/icinga2/features-enabled/api.conf
+
+ object ApiListener "api" {
+ //...
+ accept_commands = true
+ accept_config = true
+ }
+
+Now it is time to validate the configuration and restart the Icinga 2 daemon
+on both nodes.
+
+Example on CentOS 7:
+
+ [root@icinga2-client2.localdomain /]# icinga2 daemon -C
+ [root@icinga2-client2.localdomain /]# systemctl restart icinga2
+
+ [root@icinga2-master1.localdomain /]# icinga2 daemon -C
+ [root@icinga2-master1.localdomain /]# systemctl restart icinga2
+
+Once the clients have connected you are ready for the next step - **execute
+a remote check on the client using the command endpoint**.
+
+Put the host and service object configuration into the `master` zone
+-- this will help adding a secondary master for High-Availability later.
+
+ [root@icinga2-master1.localdomain /]# mkdir -p /etc/icinga2/zones.d/master
+
+Add the host and service objects you want to monitor. There is
+no limitation with files and directories -- best practice is to
+keep things organised by type.
+
+By convention a master/satellite/client host object should use the same name as the endpoint object.
+You can also add multiple hosts which execute checks against remote services/clients.
+
+ [root@icinga2-master1.localdomain /]# cd /etc/icinga2/zones.d/master
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/master]# vim hosts.conf
+
+ object Host "icinga2-client2.localdomain" {
+ check_command = "hostalive" //check is executed on the master
+ address = "192.168.56.112"
+
+ vars.client_endpoint = host.name //follows the convention host name == endpoint name
+ }
+
+Given that you are monitoring a Linux client we'll just add a remote [disk](10-icinga-template-library.md#plugin-check-command-disk)
+check.
+
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/master]# vim services.conf
+
+ apply Service "disk" {
+ check_command = "disk"
+
+ //specify where the check is executed
+ command_endpoint = host.vars.client_endpoint
+
+ assign where host.vars.client_endpoint
+ }
+
+In case that you have your own custom CheckCommand add it to the global zone.
+
+ [root@icinga2-master1.localdomain /]# mkdir -p /etc/icinga2/zones.d/global-templates
+ [root@icinga2-master1.localdomain /]# vim /etc/icinga2/zones.d/global-templates/commands.conf
+
+ object CheckCommand "my-cmd" {
+ import "plugin-check-command"
+
+ //...
+ }
+
+Save and validate the configuration on the master node.
+
+ [root@icinga2-master1.localdomain /]# icinga2 daemon -C
+
+Restart the Icinga 2 daemon (example for CentOS 7):
+
+ [root@icinga2-master1.localdomain /]# systemctl restart icinga2
+
+Now the following happens:
+
+* Icinga 2 validates the configuration on `icinga2-master1.localdomain` and restarts.
+* The `icinga2-master1.localdomain` node schedules and executes the checks.
+* The `icinga2-client2.localdomain` node receives the execute command event with additional command parameters.
+* The `icinga2-client2.localdomain` node maps the command parameters onto the local check command, executes the check locally and sends back the check result message.
+
+You'll see - no reload or any interaction required on the client
+itself.
+
+Now you have learned the basics about command endpoint checks. Proceed in
+the [scenarios](6-distributed-monitoring.md#distributed-monitoring-scenarios)
+chapter for more details on extending the setup.
+
+
+### <a id="distributed-monitoring-top-down-config-sync"></a> Top Down Config Sync
+
+This mode syncs the object configuration files within specified zones.
+This comes in handy if you want to configure everything on the master node
+and sync the satellite checks (disk, memory, etc.). The satellites run their
+own local scheduler and will send the check result messages back to the master.
+
+
+
+Advantages:
+
+* Sync the configuration files from the parent zone to the child zones.
+* No manual restart required on the child nodes - sync, validation and restarts happen automatically.
+* Execute checks directly on the child node's scheduler.
+* Replay log if the connection drops (important for keeping the check history in sync, e.g. for SLA reports).
+* Use a global zone for syncing templates, groups, etc.
+
+Disadvantages:
+
+* Requires a config directory on the master node with the zone name underneath `/etc/icinga2/zones.d`.
+* Additional zone and endpoint configuration.
+* Replay log is replicated on reconnect. This might generate an overload on the used connection.
+
+In order that all involved nodes will accept configuration and/or
+commands you'll need to configure the `Zone` and `Endpoint` hierarchy
+on all nodes.
+
+* `icinga2-master1.localdomain` is the configuration master in this scenario.
+* `icinga2-client1.localdomain` acts as client which receives configuration from the master.
+
+Put the endpoint and zone configuration on **both** nodes into `/etc/icinga2/zones.conf`.
+
+The endpoint configuration could look like this:
+
+ object Endpoint "icinga2-master1.localdomain" {
+ host = "192.168.56.101"
+ }
+
+ object Endpoint "icinga2-client1.localdomain" {
+ host = "192.168.56.111"
+ }
+
+Then you'll need to define two zones. There is no naming convention but best practice
+is to either use `master`, `satellite`/`client-fqdn` or go by region names.
+
+> **Note**
+>
+> Each client requires its own zone and endpoint configuration. Best practice
+> has been to use the client's FQDN for all object names.
+
+The `master` zone is a parent of the `icinga2-client1.localdomain` zone.
+
+ object Zone "master" {
+ endpoints = [ "icinga2-master1.localdomain" ] //array with endpoint names
+ }
+
+ object Zone "icinga2-client1.localdomain" {
+ endpoints = [ "icinga2-client1.localdomain" ]
+
+ parent = "master" //establish zone hierarchy
+ }
+
+Edit the `api` feature on the client `icinga2-client1.localdomain` in
+the `/etc/icinga2/features-enabled/api.conf` file and ensure to set
+`accept_config` to `true`.
+
+ [root@icinga2-client1.localdomain /]# vim /etc/icinga2/features-enabled/api.conf
+
+ object ApiListener "api" {
+ //...
+ accept_config = true
+ }
+
+Now it is time to validate the configuration and restart the Icinga 2 daemon
+on both nodes.
+
+Example on CentOS 7:
+
+ [root@icinga2-client1.localdomain /]# icinga2 daemon -C
+ [root@icinga2-client1.localdomain /]# systemctl restart icinga2
+
+ [root@icinga2-master1.localdomain /]# icinga2 daemon -C
+ [root@icinga2-master1.localdomain /]# systemctl restart icinga2
+
+
+> **Tip**
+>
+> Best practice is to use a [global zone](6-distributed-monitoring.md#distributed-monitoring-global-zone-config-sync)
+> for common configuration items (check commands, templates, groups, etc.).
+
+Once the clients have connected you are ready for the next step - **execute
+a local check on the client using the configuration sync**.
+
+Therefore navigate into `/etc/icinga2/zones.d` on your config master
+`icinga2-master1.localdomain` and create a new directory with the same
+name as your satellite/client zone name.
+
+ [root@icinga2-master1.localdomain /]# mkdir -p /etc/icinga2/zones.d/icinga2-client1.localdomain
+
+Add the host and service objects you want to monitor. There is
+no limitation with files and directories -- best practice is to
+keep things organised by type.
+
+By convention a master/satellite/client host object should use the same name as the endpoint object.
+You can also add multiple hosts which execute checks against remote services/clients.
+
+ [root@icinga2-master1.localdomain /]# cd /etc/icinga2/zones.d/icinga2-client1.localdomain
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/icinga2-client1.localdomain]# vim hosts.conf
+
+ object Host "icinga2-client1.localdomain" {
+ check_command = "hostalive"
+ address = "192.168.56.111"
+ zone = "master" //optional trick: sync the required host object to the client, but enforce the "master" zone to execute the check
+ }
+
+Given that you are monitoring a Linux client we'll just add a local [disk](10-icinga-template-library.md#plugin-check-command-disk)
+check.
+
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/icinga2-client1.localdomain]# vim services.conf
+
+ object Service "disk" {
+ host_name = "icinga2-client1.localdomain"
+
+ check_command = "disk"
+ }
+
+Save and validate the configuration on the master node.
+
+ [root@icinga2-master1.localdomain /]# icinga2 daemon -C
+
+Restart the Icinga 2 daemon (example for CentOS 7):
+
+ [root@icinga2-master1.localdomain /]# systemctl restart icinga2
+
+Now the following happens:
+
+* Icinga 2 validates the configuration on `icinga2-master1.localdomain`
+* Icinga 2 copies the configuration into its zone config store in `/var/lib/icinga2/api/zones`
+* The `icinga2-master1.localdomain` node sends a config update event to all endpoints in the same or direct child zones.
+* The `icinga2-client1.localdomain` node accepts config and populates the local zone config store with the received config files.
+* The `icinga2-client1.localdomain` node validates the configuration and automatically restarts.
+
+You'll see - no reload or any interaction required on the client
+itself.
+
+You can also use the config sync inside a High-Availability zone to
+ensure that all config objects are synced among zone members.
+
+> **Note**
+>
+> You can only have one so-called "config master" in a zone which stores
+> configuration in `zones.d`. Everything else breaks and is not supported.
+
+Now you have learned the basics about the configuration sync. Proceed in
+the [scenarios](6-distributed-monitoring.md#distributed-monitoring-scenarios)
+chapter for more details on extending the setup.
+
+
+### <a id="distributed-monitoring-bottom-up"></a> Bottom Up Import
+
+This mode requires you to manage the configuration on the client itself.
+Edit the configuration files underneath `/etc/icinga2/conf.d` or any other
+directory included in the `icinga2.conf` file.
+
+The client will send the configured objects to the parent zone members
+where they can generate configuration objects gathered from that information.
+
+
+
+Advantages:
+
+* Each child node comes preconfigured with the most common local checks.
+* Central management for zones, endpoints, hosts and services with configuration repository import.
+
+Disadvantages:
+
+* No object attribute sync. Parent nodes cannot filter for specific attributes in assign expressions.
+* Does not reliably work with a HA parent zone (single master preferred).
+* Configuration management of many client nodes is hard or impossible if you don't have access to them.
+
+You can list and import the configuration sent from clients on the master
+node. Example for listing all client services on the master node `icinga2-master1.localdomain`:
+
+ [root@icinga2-master1.localdomain /]# icinga2 node list
+ Node 'icinga2-client1.localdomain' (last seen: Sun Aug 14 11:19:14 2016)
+ * Host 'icinga2-client1.localdomain'
+ * Service 'disk'
+ * Service 'disk /'
+ * Service 'http'
+ * Service 'icinga'
+ * Service 'load'
+ * Service 'ping4'
+ * Service 'ping6'
+ * Service 'procs'
+ * Service 'ssh'
+ * Service 'swap'
+ * Service 'users'
+
+ Node 'DESKTOP-IHRPO96' (last seen: Sun Aug 14 11:19:14 2016)
+ * Host 'DESKTOP-IHRPO96'
+ * Service 'disk'
+ * Service 'disk C:'
+ * Service 'icinga'
+ * Service 'load'
+ * Service 'ping4'
+ * Service 'ping6'
+ * Service 'procs'
+ * Service 'swap'
+ * Service 'users'
+
+The object configuration must exist on the master node as well
+in order to receive check results from the clients. Therefore
+you'll need to invoke the `node update-config` cli command.
+
+ [root@icinga2-master1.localdomain /]# icinga2 node update-config
+ information/cli: Updating node configuration for
+ ...
+
+The generated configuration objects are located in `/etc/icinga2/repository.d`.
+If you have accidentally added specific hosts or services you can safely purge
+them from this directory and restart icinga 2.
+
+In case you want to blacklist or whitelist several hosts and/or services
+to not generate configuration on the master, use the `icinga2 node {black,white}list`
+cli commands.
+
+Example for blacklisting all `ping*` services, but allowing only `probe` host with `ping4`:
+
+ # icinga2 node blacklist add --zone "*" --host "*" --service "ping*"
+ # icinga2 node whitelist add --zone "*" --host "probe" --service "ping*"
+
+You can `list` and `remove` existing blacklists:
+
+ # icinga2 node blacklist list
+ Listing all blacklist entries:
+ blacklist filter for Node: '*' Host: '*' Service: 'ping*'.
+
+ # icinga2 node whitelist list
+ Listing all whitelist entries:
+ whitelist filter for Node: '*' Host: 'probe' Service: 'ping*'.
+
+There are certain limitations with this mode. Currently the repository
+does not sync object attributes (custom attributes, group memberships)
+from the client to the master.
+
+You can manually edit the configuration in `/etc/icinga2/repository.d`
+and fix it. That will help with additional notification apply rules
+or group memberships required for Icinga Web 2 and addons.
+
+
+## <a id="distributed-monitoring-scenarios"></a> Scenarios
+
+These examples should give you an idea how you can build your own
+distributed monitoring environment. We've seen them all in production
+environments and received feedback from our [community](https://www.icinga.org/community/get-help/)
+and [partner support](https://www.icinga.org/services/support/) channels.
+
+* Single master with clients
+* HA master with clients as command endpoint
+* Three level cluster with config HA masters, satellites receiving config sync and clients checked using command_endpoint
+
+### <a id="distributed-monitoring-master-clients"></a> Master with Clients
+
+
+
+* `icinga2-master1.localdomain` is the primary master node
+* `icinga2-client1.localdomain` and `icinga2-client2.localdomain` are two child nodes as clients
+
+Setup requirements:
+
+* Install `icinga2-master1.localdomain` as [master setup](6-distributed-monitoring.md#distributed-monitoring-setup-master)
+* Install `icinga2-client1.localdomain` and `icinga2-client2.localdomain` as [client setup](6-distributed-monitoring.md#distributed-monitoring-setup-satellite-client)
+
+Edit the `zones.conf` configuration file on the master:
+
+ [root@icinga2-master1.localdomain /]# vim /etc/icinga2/zones.conf
+
+ object Endpoint "icinga2-master1.localdomain" {
+ }
+
+ object Endpoint "icinga2-client1.localdomain" {
+ host = "192.168.33.111" //the master actively tries to connect to the client
+ }
+
+ object Endpoint "icinga2-client2.localdomain" {
+ host = "192.168.33.112" //the master actively tries to connect to the client
+ }
+
+ object Zone "master" {
+ endpoints = [ "icinga2-master1.localdomain" ]
+ }
+
+ object Zone "icinga2-client1.localdomain" {
+ endpoints = [ "icinga2-client1.localdomain" ]
+ }
+
+ object Zone "icinga2-client2.localdomain" {
+ endpoints = [ "icinga2-client2.localdomain" ]
+ }
+
+ /* sync global commands */
+ object Zone "global-templates" {
+ global = true
+ }
+
+The two client nodes do not necessarily need to know about each other. The only important thing
+is that they know about the parent zone and their endpoint members and optional the global zone.
+
+If you specify the `host` attribute in the `icinga2-master1.localdomain` endpoint object
+the client will actively try to connect to the master node. Since we've specified the client
+endpoint's attribute on the master node already, we don't want the clients to connect to the
+master. Choose one connection direction.
+
+ [root@icinga2-client1.localdomain /]# vim /etc/icinga2/zones.conf
+
+ object Endpoint "icinga2-master1.localdomain" {
+ //do not actively connect to the master by leaving out the 'host' attribute
+ }
+
+ object Endpoint "icinga2-client1.localdomain" {
+ }
+
+ object Zone "master" {
+ endpoints = [ "icinga2-master1.localdomain" ]
+ }
+
+ object Zone "icinga2-client1.localdomain" {
+ endpoints = [ "icinga2-client1.localdomain" ]
+ }
+
+ /* sync global commands */
+ object Zone "global-templates" {
+ global = true
+ }
+
+ [root@icinga2-client2.localdomain /]# vim /etc/icinga2/zones.conf
+
+ object Endpoint "icinga2-master1.localdomain" {
+ //do not actively connect to the master by leaving out the 'host' attribute
+ }
+
+ object Endpoint "icinga2-client2.localdomain" {
+ }
+
+ object Zone "master" {
+ endpoints = [ "icinga2-master1.localdomain" ]
+ }
+
+ object Zone "icinga2-client2.localdomain" {
+ endpoints = [ "icinga2-client2.localdomain" ]
+ }
+
+ /* sync global commands */
+ object Zone "global-templates" {
+ global = true
+ }
+
+Now it is time to define the two client hosts and apply service checks using
+the command endpoint execution method to them. Note: You can also use the
+config sync mode here.
+
+Create a new configuration directory on the master node.
+
+ [root@icinga2-master1.localdomain /]# mkdir -p /etc/icinga2/zones.d/master
+
+Add the two client nodes as host objects.
+
+ [root@icinga2-master1.localdomain /]# cd /etc/icinga2/zones.d/master
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/master]# vim hosts.conf
+
+ object Host "icinga2-client1.localdomain" {
+ check_command = "hostalive"
+ address = "192.168.56.111"
+ vars.client_endpoint = host.name //follows the convention host name == endpoint name
+ }
+
+ object Host "icinga2-client2.localdomain" {
+ check_command = "hostalive"
+ address = "192.168.56.112"
+ vars.client_endpoint = host.name //follows the convention host name == endpoint name
+ }
+
+Add services using command endpoint checks.
+
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/master]# vim services.conf
+
+ apply Service "ping4" {
+ check_command = "ping4"
+ //check is executed on the master node
+ assign where host.address
+ }
+
+ apply Service "disk" {
+ check_command = "disk"
+
+ //specify where the check is executed
+ command_endpoint = host.vars.client_endpoint
+
+ assign where host.vars.client_endpoint
+ }
+
+Validate the configuration and restart Icinga 2 on the master node `icinga2-master1.localdomain`.
+
+Open Icinga Web 2 and check the 2 newly created clients hosts with two new services
+-- one executed locally (`ping4`) and one using command endpoint (`disk`).
+
+### <a id="distributed-monitoring-scenarios-ha-master-clients"></a> High-Availability Master with Clients
+
+
+
+This scenario is quite the same as you have already found in the [chapter before](6-distributed-monitoring.md#distributed-monitoring-master-clients).
+
+The real difference is that we will now setup two master nodes in a High-Availablity setup.
+These nodes must be configured into zone and endpoints objects.
+
+This scenario uses the capabilities of the Icinga 2 cluster. All zone members
+replicate cluster events amongst each other. In addition to that several Icinga 2
+features can enable HA functionality.
+
+> **Notes**
+> All nodes in the same zone require the same features enabled for High Availability (HA)
+> amongst them.
+
+Overview:
+
+* `icinga2-master1.localdomain` is the config master master node
+* `icinga2-master2.localdomain` is the secondary master master node without config in `zones.d`
+* `icinga2-client1.localdomain` and `icinga2-client2.localdomain` are two child nodes as clients
+
+Setup requirements:
+
+* Install `icinga2-master1.localdomain` as [master setup](6-distributed-monitoring.md#distributed-monitoring-setup-master)
+* Install `icinga2-master2.localdomain` as [client setup](6-distributed-monitoring.md#distributed-monitoring-setup-satellite-client) (we will modify the generated configuration)
+* Install `icinga2-client1.localdomain` and `icinga2-client2.localdomain` as [client setup](6-distributed-monitoring.md#distributed-monitoring-setup-satellite-client) (when asked for adding multiple masters, tick 'y' and add the secondary master `icinga2-master2.localdomain`).
+
+In case you not want to use the cli commands you can also manually create and sync the
+required SSL certificates. We will modify and discuss the generated configuration here
+in detail.
+
+Since there are now two nodes in the same zone we must consider the
+[high-availability features](6-distributed-monitoring.md#distributed-monitoring-high-availability-features).
+
+* Checks and notifiations are balanced between the two master nodes. That's fine but requires check plugins and notification scripts to exist on both nodes.
+* The IDO feature will only be active on one node by default. Since all events are replicated between both nodes it is easier to just have one central database.
+
+Decide whether you want to use a dedicated MySQL cluster VIP (external application cluster)
+and leave the IDO feature with enabled HA capabilities. Or you'll configure the feature to
+disable HA and write to a local installed database on each node. Both implementation methods
+require you to configure Icinga Web 2 accordingly (Monitoring backend - IDO database, used transports).
+
+The zone hierarchy could look like this. It involves putting the two master nodes
+`icinga2-master1.localdomain` and `icinga2-master2.localdomain` into the `master` zone.
+
+ [root@icinga2-master1.localdomain /]# vim /etc/icinga2/zones.conf
+
+ object Endpoint "icinga2-master1.localdomain" {
+ host = "192.168.56.101"
+ }
+
+ object Endpoint "icinga2-master2.localdomain" {
+ host = "192.168.56.101"
+ }
+
+ object Endpoint "icinga2-client1.localdomain" {
+ host = "192.168.33.111" //the master actively tries to connect to the client
+ }
+
+ object Endpoint "icinga2-client2.localdomain" {
+ host = "192.168.33.112" //the master actively tries to connect to the client
+ }
+
+ object Zone "master" {
+ endpoints = [ "icinga2-master1.localdomain", "icinga2-master1.localdomain" ]
+ }
+
+ object Zone "icinga2-client1.localdomain" {
+ endpoints = [ "icinga2-client1.localdomain" ]
+ }
+
+ object Zone "icinga2-client2.localdomain" {
+ endpoints = [ "icinga2-client2.localdomain" ]
+ }
+
+ /* sync global commands */
+ object Zone "global-templates" {
+ global = true
+ }
+
+The two client nodes do not necessarily need to know about each other. The only important thing
+is that they know about the parent zone and their endpoint members and optional the global zone.
+
+If you specify the `host` attribute in the `icinga2-master1.localdomain` and `icinga2-master2.localdomain`
+endpoint objects the client will actively try to connect to the master node. Since we've specified the client
+endpoint's attribute on the master node already, we don't want the clients to connect to the
+master nodes. Choose one connection direction.
+
+ [root@icinga2-client1.localdomain /]# vim /etc/icinga2/zones.conf
+
+ object Endpoint "icinga2-master1.localdomain" {
+ //do not actively connect to the master by leaving out the 'host' attribute
+ }
+
+ object Endpoint "icinga2-master2.localdomain" {
+ //do not actively connect to the master by leaving out the 'host' attribute
+ }
+
+ object Endpoint "icinga2-client1.localdomain" {
+ }
+
+ object Zone "master" {
+ endpoints = [ "icinga2-master1.localdomain", "icinga2-master2.localdomain" ]
+ }
+
+ object Zone "icinga2-client1.localdomain" {
+ endpoints = [ "icinga2-client1.localdomain" ]
+ }
+
+ /* sync global commands */
+ object Zone "global-templates" {
+ global = true
+ }
+
+ [root@icinga2-client2.localdomain /]# vim /etc/icinga2/zones.conf
+
+ object Endpoint "icinga2-master1.localdomain" {
+ //do not actively connect to the master by leaving out the 'host' attribute
+ }
+
+ object Endpoint "icinga2-master2.localdomain" {
+ //do not actively connect to the master by leaving out the 'host' attribute
+ }
+
+ object Endpoint "icinga2-client2.localdomain" {
+ }
+
+ object Zone "master" {
+ endpoints = [ "icinga2-master1.localdomain", "icinga2-master2.localdomain" ]
+ }
+
+ object Zone "icinga2-client2.localdomain" {
+ endpoints = [ "icinga2-client2.localdomain" ]
+ }
+
+ /* sync global commands */
+ object Zone "global-templates" {
+ global = true
+ }
+
+Now it is time to define the two client hosts and apply service checks using
+the command endpoint execution method to them. Note: You can also use the
+config sync mode here.
+
+Create a new configuration directory on the master node `icinga2-master1.localdomain`.
+**Note**: The secondary master node `icinga2-master2.localdomain` receives the
+configuration using the [config sync mode](6-distributed-monitoring.md#distributed-monitoring-top-down-config-sync).
+
+ [root@icinga2-master1.localdomain /]# mkdir -p /etc/icinga2/zones.d/master
+
+Add the two client nodes as host objects.
+
+ [root@icinga2-master1.localdomain /]# cd /etc/icinga2/zones.d/master
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/master]# vim hosts.conf
+
+ object Host "icinga2-client1.localdomain" {
+ check_command = "hostalive"
+ address = "192.168.56.111"
+ vars.client_endpoint = host.name //follows the convention host name == endpoint name
+ }
+
+ object Host "icinga2-client2.localdomain" {
+ check_command = "hostalive"
+ address = "192.168.56.112"
+ vars.client_endpoint = host.name //follows the convention host name == endpoint name
+ }
+
+Add services using command endpoint checks.
+
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/master]# vim services.conf
+
+ apply Service "ping4" {
+ check_command = "ping4"
+ //check is executed on the master node
+ assign where host.address
+ }
+
+ apply Service "disk" {
+ check_command = "disk"
+
+ //specify where the check is executed
+ command_endpoint = host.vars.client_endpoint
+
+ assign where host.vars.client_endpoint
+ }
+
+Validate the configuration and restart Icinga 2 on the master node `icinga2-master1.localdomain`.
+
+Open Icinga Web 2 and check the 2 newly created clients hosts with two new services
+-- one executed locally (`ping4`) and one using command endpoint (`disk`).
+
+In addition to that you should add [health checks](6-distributed-monitoring.md#distributed-monitoring-health-checks)
+ensuring that your cluster notifies you in case of failure.
+
+
+### <a id="distributed-monitoring-scenarios-master-satellite-client"></a> Three Levels with Master, Satellites and Clients
+
+
+
+This scenario combines everything you've learned so far. High-availability masters,
+satellites receiving their config from the master zone, clients checked via command
+endpoint from the satellite zones.
+
+> **Tip**
+>
+> It can get complicated so take pen and paper and bring your thoughts to life.
+> Play around with a test setup before putting such a thing into production too!
+
+Overview:
+
+* `icinga2-master1.localdomain` is the config master master node
+* `icinga2-master2.localdomain` is the secondary master master node without config in `zones.d`
+* `icinga2-satellite1.localdomain` and `icinga2-satellite2.localdomain` are satellite nodes in a `master` child zone
+* `icinga2-client1.localdomain` and `icinga2-client2.localdomain` are two child nodes as clients
+
+Setup requirements:
+
+* Install `icinga2-master1.localdomain` as [master setup](6-distributed-monitoring.md#distributed-monitoring-setup-master)
+* Install `icinga2-master2.localdomain`, `icinga2-satellite1.localdomain` and `icinga2-satellite2.localdomain` as [client setup](6-distributed-monitoring.md#distributed-monitoring-setup-satellite-client) (we will modify the generated configuration)
+* Install `icinga2-client1.localdomain` and `icinga2-client2.localdomain` as [client setup](6-distributed-monitoring.md#distributed-monitoring-setup-satellite-client)
+
+Once you are asked for the master endpoint providing CSR auto-signing capabilities
+please add the master node which holds the CA and has the ApiListener feature configured.
+The parent endpoint must still remain the satellite endpoint name.
+
+Example for `icinga2-client1.localdomain`:
+
+ Please specify the master endpoint(s) this node should connect to:
+
+"master" is the first satellite `icinga2-satellite1.localdomain`.
+
+ Master Common Name (CN from your master setup): icinga2-satellite1.localdomain
+ Do you want to establish a connection to the master from this node? [Y/n]: y
+ Please fill out the master connection information:
+ Master endpoint host (Your master's IP address or FQDN): 192.168.56.105
+ Master endpoint port [5665]:
+
+Add more "masters", the second satellite `icinga2-satellite2.localdomain`.
+
+ Add more master endpoints? [y/N]: y
+ Master Common Name (CN from your master setup): icinga2-satellite2.localdomain
+ Do you want to establish a connection to the master from this node? [Y/n]: y
+ Please fill out the master connection information:
+ Master endpoint host (Your master's IP address or FQDN): 192.168.56.106
+ Master endpoint port [5665]:
+ Add more master endpoints? [y/N]: n
+
+Specify the master node `icinga2-master2.localdomain`with the CA private key and ticket salt configured.
+
+ Please specify the master connection for CSR auto-signing (defaults to master endpoint host):
+ Host [192.168.56.106]: icinga2-master1.localdomain
+ Port [5665]:
+
+In case you cannot connect to the master node from your clients, you'll manually need
+to [generate the SSL certificates](6-distributed-monitoring.md#distributed-monitoring-advanced-hints-certificates)
+and modify the configuration.
+
+We'll discuss the required configuration in detail below.
+
+The zone hierarchy can look like this. We'll define only the directly connected zones here.
+
+You can safely deploy this configuration onto all master and satellite zone
+members. You should keep in mind to control the endpoint connection direction
+using the `host` attribute.
+
+ [root@icinga2-master1.localdomain /]# vim /etc/icinga2/zones.conf
+
+ object Endpoint "icinga2-master1.localdomain" {
+ host = "192.168.56.101"
+ }
+
+ object Endpoint "icinga2-master2.localdomain" {
+ host = "192.168.56.101"
+ }
+
+ object Endpoint "icinga2-satellite1.localdomain" {
+ host = "192.168.56.105"
+ }
+
+ object Endpoint "icinga2-satellite2.localdomain" {
+ host = "192.168.56.106"
+ }
+
+ object Zone "master" {
+ endpoints = [ "icinga2-master1.localdomain", "icinga2-master1.localdomain" ]
+ }
+
+ object Zone "satellite" {
+ endpoints = [ "icinga2-satellite1.localdomain", "icinga2-satellite1.localdomain" ]
+ }
+
+ /* sync global commands */
+ object Zone "global-templates" {
+ global = true
+ }
+
+Note: The master nodes do not need to know about the indirectly connected clients
+for connection reasons. But since we want to use command endpoint check configuration,
+we'll need them. In order to maximize the effort, we'll sync the client zone and endpoint
+config to the satellites where the connection information is needed as well.
+
+ [root@icinga2-master1.localdomain /]# mkdir -p /etc/icinga2/zones.d/{master,satellite,global-templates}
+ [root@icinga2-master1.localdomain /]# cd /etc/icinga2/zones.d/satellite
+
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/satellite]# vim icinga2-client1.localdomain.conf
+
+ object Endpoint "icinga2-client1.localdomain" {
+ host = "192.168.33.111" //the satellite actively tries to connect to the client
+ }
+
+ object Zone "icinga2-client1.localdomain" {
+ endpoints = [ "icinga2-client1.localdomain" ]
+ }
+
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/satellite]# vim icinga2-client2.localdomain.conf
+
+ object Endpoint "icinga2-client2.localdomain" {
+ host = "192.168.33.112" //the satellite actively tries to connect to the client
+ }
+
+ object Zone "icinga2-client2.localdomain" {
+ endpoints = [ "icinga2-client2.localdomain" ]
+ }
+
+The two client nodes themselves do not necessarily need to know about each other. The only important thing
+is that they know about the parent zone and their endpoint members and optional the global zone.
+
+If you specify the `host` attribute in the `icinga2-master1.localdomain` and `icinga2-master2.localdomain`
+endpoint objects the client will actively try to connect to the master node. Since we've specified the client
+endpoint's attribute on the master node already, we don't want the clients to connect to the
+master nodes. Choose one connection direction.
+
+ [root@icinga2-client1.localdomain /]# vim /etc/icinga2/zones.conf
+
+ object Endpoint "icinga2-satellite1.localdomain" {
+ //do not actively connect to the satellite by leaving out the 'host' attribute
+ }
+
+ object Endpoint "icinga2-satellite2.localdomain" {
+ //do not actively connect to the satellite by leaving out the 'host' attribute
+ }
+
+ object Endpoint "icinga2-client1.localdomain" {
+ }
+
+ object Zone "satellite" {
+ endpoints = [ "icinga2-satellite1.localdomain", "icinga2-satellite2.localdomain" ]
+ }
+
+ object Zone "icinga2-client1.localdomain" {
+ endpoints = [ "icinga2-client1.localdomain" ]
+ }
+
+ /* sync global commands */
+ object Zone "global-templates" {
+ global = true
+ }
+
+ [root@icinga2-client2.localdomain /]# vim /etc/icinga2/zones.conf
+
+ object Endpoint "icinga2-satellite1.localdomain" {
+ //do not actively connect to the satellite by leaving out the 'host' attribute
+ }
+
+ object Endpoint "icinga2-satellite2.localdomain" {
+ //do not actively connect to the satellite by leaving out the 'host' attribute
+ }
+
+ object Endpoint "icinga2-client2.localdomain" {
+ }
+
+ object Zone "satellite" {
+ endpoints = [ "icinga2-satellite1.localdomain", "icinga2-satellite2.localdomain" ]
+ }
+
+ object Zone "icinga2-client2.localdomain" {
+ endpoints = [ "icinga2-client2.localdomain" ]
+ }
+
+ /* sync global commands */
+ object Zone "global-templates" {
+ global = true
+ }
+
+Now it is time to define the two client hosts on the master, sync them to the satellites
+ and apply service checks using the command endpoint execution method to them.
+
+Add the two client nodes as host objects into the `satellite` zone.
+**Note**: We've previously created the directories in `zones.d` and files for the
+zone and endpoint configuration for the clients already.
+
+ [root@icinga2-master1.localdomain /]# cd /etc/icinga2/zones.d/satellite
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/satellite]# vim icinga2-client1.localdomain.conf
+
+ object Host "icinga2-client1.localdomain" {
+ check_command = "hostalive"
+ address = "192.168.56.111"
+ vars.client_endpoint = host.name //follows the convention host name == endpoint name
+ }
+
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/satellite]# vim icinga2-client2.localdomain.conf
+
+ object Host "icinga2-client2.localdomain" {
+ check_command = "hostalive"
+ address = "192.168.56.112"
+ vars.client_endpoint = host.name //follows the convention host name == endpoint name
+ }
+
+Add services using command endpoint checks. Pin the apply rules to the `satellite` zone only.
+
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/satellite]# vim services.conf
+
+ apply Service "ping4" {
+ check_command = "ping4"
+ //check is executed on the satellite node
+ assign where host.zone == "satellite" && host.address
+ }
+
+ apply Service "disk" {
+ check_command = "disk"
+
+ //specify where the check is executed
+ command_endpoint = host.vars.client_endpoint
+
+ assign where host.zone == "satellite" && host.vars.client_endpoint
+ }
+
+Validate the configuration and restart Icinga 2 on the master node `icinga2-master1.localdomain`.
+
+Open Icinga Web 2 and check the 2 newly created clients hosts with two new services
+-- one executed locally (`ping4`) and one using command endpoint (`disk`).
+
+In addition to that you should add [health checks](6-distributed-monitoring.md#distributed-monitoring-health-checks)
+ensuring that your cluster notifies you in case of failure.
+
+
+## <a id="distributed-monitoring-best-practice"></a> Best Practice
+
+A collection of best practices we've learned from the community.
+
+Join the [community channels](https://www.icinga.org/community/get-help/)
+and share your tips and tricks with us!
+
+### <a id="distributed-monitoring-global-zone-config-sync"></a> Global Zone for Config Sync
+
+The idea behind a global zone is not to add endpoints to it. That would
+not work with the implemented cluster hierarchy.
+
+It was rather designed with the problem in mind - the configuration synced
+to each node must be valid. What if my templates and check commands are
+only available on the master node?
+
+Therefore it is possible to use the config sync mode with a global zone.
+
+The zone object configuration must be deployed on all nodes which should receive
+the global configuration files.
+
+ object Zone "global-templates" {
+ global = true
+ }
+
+Similar to the zone configuration sync you'll need to create a new directory in
+`/etc/icinga2/zones.d`
+
+ [root@icinga2-master1.localdomain /]# mkdir -p /etc/icinga2/zones.d/global-templates
+
+Then add a new check command for example.
+
+ [root@icinga2-master1.localdomain /]# vim /etc/icinga2/zones.d/global-templates/commands.conf
+
+ object CheckCommand "my-cmd" {
+ import "plugin-check-command"
+
+ //...
+ }
+
+Restart the client(s) which should receive the global zone first.
+
+Then validate the configuration on the master node and restart Icinga 2.
+
+**Note**: Host/Service objects must not be put into a global zone. The configuration
+validation will throw an error.
+
+### <a id="distributed-monitoring-health-checks"></a> Health Checks
+
+In case of network failures or any other problem your monitoring could
+either have late check results or just send out mass alarms for unknown
+checks.
+
+In order to minimize the problems caused by this you should configure
+additional health checks.
+
+The `cluster` check will check if all endpoints in the current zone and the directly
+connected zones are working properly.
+
+ object Service "cluster" {
+ check_command = "cluster"
+ check_interval = 5s
+ retry_interval = 1s
+
+ host_name = "icinga2-master1.localdomain"
+ }
+
+The `cluster-zone` check will test whether the configured target zone is currently
+connected or not.
+
+ apply Service "child-health" {
+ check_command = "cluster-zone"
+
+ /* This follows the convention that the client zone name is the FQDN which is the same as the host object name. */
+ vars.cluster_zone = host.name
+
+ assign where host.vars.has_client
+ }
+
+In case you cannot assign the `cluster_zone` attribute that generic add specific
+checks to your cluster.
+
+ object Service "cluster-zone-satellite" {
+ check_command = "cluster-zone"
+ check_interval = 5s
+ retry_interval = 1s
+ vars.cluster_zone = "satellite"
+
+ host_name = "icinga2-master1.localdomain"
+ }
+
+
+In case you are using top down checks with command endpoint configuration you can
+for example add a dependency which prevents notifications for all other failing services:
+
+ apply Dependency "health-check" to Service {
+ parent_service_name = "child-health"
+
+ states = [ OK ]
+ disable_notifications = true
+
+ assign where host.vars.has_client
+ ignore where service.name == "child-health"
+ }
+
+### <a id="distributed-monitoring-windows-plugins"></a> Windows Client and Plugins
+
+The Icinga 2 package on Windows already provides several plugins. There is
+a detailed documentation for all available check command definitions over [here](10-icinga-template-library.md#windows-plugins).
+
+Add the following inclusion on all your nodes (master, satellite, client):
+
+ vim /etc/icinga2/icinga2.conf
+
+ include <windows-plugins>
+
+Based on the [master with clients](6-distributed-monitoring.md#distributed-monitoring-master-clients)
+scenario we'll now add a local disk check.
+
+Add the client node as host object.
+
+ [root@icinga2-master1.localdomain /]# cd /etc/icinga2/zones.d/master
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/master]# vim hosts.conf
+
+ object Host "icinga2-client1.localdomain" {
+ check_command = "hostalive"
+ address = "192.168.56.111"
+ vars.client_endpoint = host.name //follows the convention host name == endpoint name
+ vars.os_type = "windows"
+ }
+
+Add the disk check using command endpoint checks (details in the
+[disk-windows](10-icinga-template-library.md#windows-plugins-disk-windows) documentation).
+
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/master]# vim services.conf
+
+ apply Service "disk C:" {
+ check_command = "disk-windows"
+
+ vars.disk_win_path = "C:"
+
+ //specify where the check is executed
+ command_endpoint = host.vars.client_endpoint
+
+ assign where host.vars.os_type == "windows" && host.vars.client_endpoint
+ }
+
+
+### <a id="distributed-monitoring-windows-nscp"></a> Windows Client and NSClient++
+
+The [Windows setup](#distributed-monitoring-setup-client-windows) already allows
+you to install the NSClient++ package. In addition to the Windows plugins you can
+also use the [nscp-local commands](10-icinga-template-library.md#nscp-plugin-check-commands)
+provided by the Icinga Template Library (ITL).
+
+
+
+Add the following inclusion on all your nodes (master, satellite, client):
+
+ vim /etc/icinga2/icinga2.conf
+
+ include <nscp-local>
+
+The CheckCommand definitions will automatically determine the installed path
+to the `nscp.exe` binary.
+
+Based on the [master with clients](6-distributed-monitoring.md#distributed-monitoring-master-clients)
+scenario we'll now add a local nscp check querying a given performance counter.
+
+Add the client node as host object.
+
+ [root@icinga2-master1.localdomain /]# cd /etc/icinga2/zones.d/master
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/master]# vim hosts.conf
+
+ object Host "icinga2-client1.localdomain" {
+ check_command = "hostalive"
+ address = "192.168.56.111"
+ vars.client_endpoint = host.name //follows the convention host name == endpoint name
+ }
+
+Add performance counter check using command endpoint checks (details in the
+[nscp-local-counter](10-icinga-template-library.md#nscp-check-local-counter) documentation).
+
+ [root@icinga2-master1.localdomain /etc/icinga2/zones.d/master]# vim services.conf
+
+ apply Service "perf-counter-cpu" {
+ check_command = "nscp-local-counter"
+
+ vars.nscp_local_counter = "\\Processor(_total)\\% Processor Time"
+ vars.nscp_local_perfsyntax = "Total Processor Time"
+ vars.nscp_local_warning = 1
+ vars.nscp_local_critical = 5
+
+ //specify where the check is executed
+ command_endpoint = host.vars.client_endpoint
+
+ assign where host.vars.client_endpoint
+ }
+
+
+## <a id="distributed-monitoring-advanced-hints"></a> Advanced Hints
+
+You can find additional hints in this section if you prefer to go your own route
+with automating setups (setup, certificates, configuration).
+
+### <a id="distributed-monitoring-high-availability-features"></a> High Availability for Icinga 2 features
+
+All nodes in the same zone require the same features enabled for High Availability (HA)
+amongst them.
+
+By default the following features provide advanced HA functionality:
+
+* [Checks](6-distributed-monitoring.md#distributed-monitoring-high-availability-checks) (load balanced, automated failover)
+* [Notifications](6-distributed-monitoring.md#distributed-monitoring-high-availability-notifications) (load balanced, automated failover)
+* [DB IDO](6-distributed-monitoring.md#distributed-monitoring-high-availability-db-ido) (Run-Once, automated failover)
+
+#### <a id="distributed-monitoring-high-availability-checks"></a> High Availability with Checks
+
+All instances within the same zone (e.g. the `master` zone as HA cluster) must
+have the `checker` feature enabled.
+
+Example:
+
+ # icinga2 feature enable checker
+
+All nodes in the same zone load-balance the check execution. When one instance shuts down
+the other nodes will automatically take over the remaining checks.
+
+#### <a id="distributed-monitoring-high-availability-notifications"></a> High Availability with Notifications
+
+All instances within the same zone (e.g. the `master` zone as HA cluster) must
+have the `notification` feature enabled.
+
+Example:
+
+ # icinga2 feature enable notification
+
+Notifications are load balanced amongst all nodes in a zone. By default this functionality
+is enabled.
+If your nodes should notify independent from any other nodes (this will cause
+duplicated notifications if not properly handled!), you can set `enable_ha = false`
+in the [NotificationComponent](9-object-types.md#objecttype-notificationcomponent) feature.
+
+#### <a id="distributed-monitoring-high-availability-db-ido"></a> High Availability with DB IDO
+
+All instances within the same zone (e.g. the `master` zone as HA cluster) must
+have the DB IDO feature enabled.
+
+Example DB IDO MySQL:
+
+ # icinga2 feature enable ido-mysql
+
+By default the DB IDO feature only runs on one node. All other nodes in the same zone disable
+the active IDO database connection at runtime. The node with the active DB IDO connection is
+not necessarily the zone master.
+
+> **Note**
+>
+> The DB IDO HA feature can be disabled by setting the `enable_ha` attribute to `false`
+> for the [IdoMysqlConnection](9-object-types.md#objecttype-idomysqlconnection) or
+> [IdoPgsqlConnection](9-object-types.md#objecttype-idopgsqlconnection) object on **all** nodes in the
+> **same** zone.
+>
+> All endpoints will enable the DB IDO feature and connect to the configured
+> database and dump configuration, status and historical data on their own.
+
+If the instance with the active DB IDO connection dies, the HA functionality will
+automatically elect a new DB IDO master.
+
+The DB IDO feature will try to determine which cluster endpoint is currently writing
+to the database and bail out if another endpoint is active. You can manually verify that
+by running the following query:
+
+ icinga=> SELECT status_update_time, endpoint_name FROM icinga_programstatus;
+ status_update_time | endpoint_name
+ ------------------------+---------------
+ 2014-08-15 15:52:26+02 | icinga2a
+ (1 Zeile)
+
+This is useful when the cluster connection between endpoints breaks, and prevents
+data duplication in split-brain-scenarios. The failover timeout can be set for the
+`failover_timeout` attribute, but not lower than 60 seconds.
+
+
+### <a id="distributed-monitoring-advanced-hints-windows-silent"></a> Silent Windows Setup
+
+If you want to install the client silently/unattended, use the `/qn` modifier. The
+installation should not trigger a restart but if you want to be completly sure you can use the `/norestart` modifier.
+
+ C:> msiexec /i C:\Icinga2-v2.5.0-x86.msi /qn /norestart
+
+### <a id="distributed-monitoring-advanced-hints-certificates"></a> Manual Certificate Creation
+
+Choose the host which should store the certificate authority (one of the master nodes).
+
+The first step is the creation of the certificate authority (CA) by running the following command
+as root user:
+
+ icinga2 pki new-ca
+
+Create a certificate signing request (CSR) for each node.
+
+ icinga2 pki new-cert --cn icinga2-master1.localdomain --key icinga2-master1.localdomain.key --csr icinga2-master1.localdomain.csr
+
+Sign the CSR with the previously created CA.
+
+ icinga2 pki sign-csr --csr icinga2-master1.localdomain.csr --cert icinga2-master1.localdomain
+
+Copy the host's certificate files and the public CA certificate to `/etc/icinga2/pki`.
+
+ mkdir -p /etc/icinga2/pki
+ cp icinga2-master1.localdomain.{crt,key} /etc/icinga2/pki
+ cp /var/lib/icinga2/ca/ca.crt /etc/icinga2/pki
+
+Ensure that proper permissions are set (replace `icinga` with the Icinga 2 daemon user).
+
+ chown -R icinga:icinga /etc/icinga2/pki
+ chmod 600 /etc/icinga2/pki/*.key
+ chmod 644 /etc/icinga2/pki/*.crt
+
+The CA public and private key are stored in the `/var/lib/icinga2/ca` directory. Keep this path secure and include
+it in your backups.
+
+Example for creating multiple certificates at once:
+
+ # for node in icinga2-master1.localdomain icinga2-master2.localdomain icinga2-satellite1.localdomain; do sudo icinga2 pki new-cert --cn $node --csr $node.csr --key $node.key; done
+ information/base: Writing private key to 'icinga2-master1.localdomain.key'.
+ information/base: Writing certificate signing request to 'icinga2-master1.localdomain.csr'.
+ information/base: Writing private key to 'icinga2-master2.localdomain.key'.
+ information/base: Writing certificate signing request to 'icinga2-master2.localdomain.csr'.
+ information/base: Writing private key to 'icinga2-satellite1.localdomain.key'.
+ information/base: Writing certificate signing request to 'icinga2-satellite1.localdomain.csr'.
+
+ # for node in icinga2-master1.localdomain icinga2-master2.localdomain icinga2-satellite1.localdomain; do sudo icinga2 pki sign-csr --csr $node.csr --cert $node.crt; done
+ information/pki: Writing certificate to file 'icinga2-master1.localdomain.crt'.
+ information/pki: Writing certificate to file 'icinga2-master2.localdomain.crt'.
+ information/pki: Writing certificate to file 'icinga2-satellite1.localdomain.crt'.
+
+### <a id="distributed-monitoring-advanced-hints-cli-node-setup"></a> Node Setup Cli Command
+
+Instead of using the `node wizard` cli command, there is an alternative `node setup`
+cli command available which has some pre-requisites. Make sure that the
+`/etc/icinga2/pki` exists and is owned by the `icinga` user (or the user Icinga 2 is
+running as).
+
+Required information:
+
+* The client common name (CN). Use the FQDN, e.g. `icinga2-node2.localdomain`.
+* The master host and zone name. Pass that to `pki save-cert` as `--host` parameter for example.
+ * Optional: Master endpoint host and port information for the `--endpoint` parameter.
+* The client ticket number generated on the master (`icinga2 pki ticket --cn icinga2-node2.localdomain`)
+
+Generate a new local self-signed certificate.
+
+ # icinga2 pki new-cert --cn icinga2-node2.localdomain \
+ --key /etc/icinga2/pki/icinga2-node2.localdomain.key \
+ --cert /etc/icinga2/pki/icinga2-node2.localdomain.crt
+
+Request the master certificate from the master host (`icinga2-node1.localdomain`)
+and store it as `trusted-master.crt`. Review it and continue.
+
+ # icinga2 pki save-cert --key /etc/icinga2/pki/icinga2-node2.localdomain.key \
+ --cert /etc/icinga2/pki/icinga2-node2.localdomain.crt \
+ --trustedcert /etc/icinga2/pki/trusted-master.crt \
+ --host icinga2-node1.localdomain
+
+Send the self-signed certificate to the master host using the ticket number and
+receive a CA signed certificate and the master's `ca.crt` certificate.
+Specify the path to the previously stored trusted master certificate.
+
+ # icinga2 pki request --host icinga2-node1.localdomain \
+ --port 5665 \
+ --ticket ead2d570e18c78abf285d6b85524970a0f69c22d \
+ --key /etc/icinga2/pki/icinga2-node2.localdomain.key \
+ --cert /etc/icinga2/pki/icinga2-node2.localdomain.crt \
+ --trustedcert /etc/icinga2/pki/trusted-master.crt \
+ --ca /etc/icinga2/pki/ca.crt
+
+Continue with the additional node setup steps. Specify a local endpoint and zone name (`icinga2-node2.localdomain`)
+and set the master host (`icinga2-node1.localdomain`) as parent zone configuration. Specify the path to
+the previously stored trusted master certificate.
+
+ # icinga2 node setup --ticket ead2d570e18c78abf285d6b85524970a0f69c22d \
+ --endpoint icinga2-node1.localdomain \
+ --zone icinga2-node2.localdomain \
+ --master_host icinga2-node1.localdomain \
+ --trustedcert /etc/icinga2/pki/trusted-master.crt
+
+In case the client should connect to the master node, you'll
+need to modify the `--endpoint` parameter using the format `cn,host,port`.
+
+ --endpoint icinga2-node1.localdomain,192.168.56.101,5665
+
+Restart Icinga 2 once complete.
+
+ # service icinga2 restart
+
The SNMP daemon runs on the remote system and answers SNMP queries by plugin
binaries. The [Monitoring Plugins package](2-getting-started.md#setting-up-check-plugins) ships
-the `check_snmp` plugin binary, but there are plenty of [existing plugins](14-addons-plugins.md#plugins)
+the `check_snmp` plugin binary, but there are plenty of [existing plugins](5-service-monitoring.md#service-monitoring-plugins)
for specific use cases already around, for example monitoring Cisco routers.
-The following example uses the [SNMP ITL](7-icinga-template-library.md#plugin-check-command-snmp) `CheckCommand` and just
+The following example uses the [SNMP ITL](10-icinga-template-library.md#plugin-check-command-snmp) `CheckCommand` and just
overrides the `snmp_oid` custom attribute. A service is created for all hosts which
have the `snmp-community` custom attribute.
assign where host.vars.snmp_community != ""
}
-Additional SNMP plugins are available using the [Manubulon SNMP Plugins](7-icinga-template-library.md#snmp-manubulon-plugin-check-commands).
+Additional SNMP plugins are available using the [Manubulon SNMP Plugins](10-icinga-template-library.md#snmp-manubulon-plugin-check-commands).
If no `snmp_miblist` is specified, the plugin will default to `ALL`. As the number of available MIB files
on the system increases so will the load generated by this plugin if no `MIB` is specified.
vars.by_ssh_logname = "icinga"
}
+## <a id="agent-based-checks-nsclient"></a> NSClient++
+
+[NSClient++](http://nsclient.org) works on both Windows and Linux platforms and is well
+known for its magnificent Windows support. There are alternatives like the WMI interface,
+but using `NSClient++` will allow you to run local scripts similar to check plugins fetching
+the required output and performance counters.
+
+You can use the `check_nt` plugin from the Monitoring Plugins project to query NSClient++.
+Icinga 2 provides the [nscp check command](10-icinga-template-library.md#plugin-check-command-nscp) for this:
+
+Example:
+
+ object Service "disk" {
+ import "generic-service"
+
+ host_name = "remote-windows-host"
+
+ check_command = "nscp"
+
+ vars.nscp_variable = "USEDDISKSPACE"
+ vars.nscp_params = "c"
+ vars.nscp_warn = 70
+ vars.nscp_crit = 80
+ }
+
+For details on the `NSClient++` configuration please refer to the [official documentation](http://www.nsclient.org/nscp/wiki/doc/configuration/0.4.x).
+
+## <a id="agent-based-checks-nsca-ng"></a> NSCA-NG
+
+[NSCA-ng](http://www.nsca-ng.org) provides a client-server pair that allows the
+remote sender to push check results into the Icinga 2 `ExternalCommandListener`
+feature.
+
+> **Note**
+>
+> This addon works in a similar fashion like the Icinga 1.x distributed model. If you
+> are looking for a real distributed architecture with Icinga 2, scroll down.
+
## <a id="agent-based-checks-nrpe"></a> NRPE
[NRPE](http://docs.icinga.org/latest/en/nrpe.html) runs as daemon on the remote client including
> The NRPE protocol is considered insecure and has multiple flaws in its
> design. Upstream is not willing to fix these issues.
>
-> In order to stay safe, please use the native [Icinga 2 client](11-icinga2-client.md#icinga2-client)
+> In order to stay safe, please use the native [Icinga 2 client](6-distributed-monitoring.md#distributed-monitoring)
> instead.
The NRPE daemon uses its own configuration format in nrpe.cfg while `check_nrpe`
can be embedded into the Icinga 2 `CheckCommand` configuration syntax.
You can use the `check_nrpe` plugin from the NRPE project to query the NRPE daemon.
-Icinga 2 provides the [nrpe check command](7-icinga-template-library.md#plugin-check-command-nrpe) for this:
+Icinga 2 provides the [nrpe check command](10-icinga-template-library.md#plugin-check-command-nrpe) for this:
Example:
/usr/local/icinga/libexec/check_disk -w 20% -c 10% -p /
-You can pass arguments in a similar manner to [NSClient++](12-agent-based-checks.md#agent-based-checks-nsclient)
+You can pass arguments in a similar manner to [NSClient++](7-agent-based-monitoring.md#agent-based-checks-nsclient)
when using its NRPE supported check method.
-## <a id="agent-based-checks-nsclient"></a> NSClient++
-
-[NSClient++](http://nsclient.org) works on both Windows and Linux platforms and is well
-known for its magnificent Windows support. There are alternatives like the WMI interface,
-but using `NSClient++` will allow you to run local scripts similar to check plugins fetching
-the required output and performance counters.
-
-You can use the `check_nt` plugin from the Monitoring Plugins project to query NSClient++.
-Icinga 2 provides the [nscp check command](7-icinga-template-library.md#plugin-check-command-nscp) for this:
-
-Example:
-
- object Service "disk" {
- import "generic-service"
-
- host_name = "remote-windows-host"
-
- check_command = "nscp"
-
- vars.nscp_variable = "USEDDISKSPACE"
- vars.nscp_params = "c"
- vars.nscp_warn = 70
- vars.nscp_crit = 80
- }
-
-For details on the `NSClient++` configuration please refer to the [official documentation](http://www.nsclient.org/nscp/wiki/doc/configuration/0.4.x).
-
-## <a id="agent-based-checks-nsca-ng"></a> NSCA-NG
-
-[NSCA-ng](http://www.nsca-ng.org) provides a client-server pair that allows the
-remote sender to push check results into the Icinga 2 `ExternalCommandListener`
-feature.
-
-> **Note**
->
-> This addon works in a similar fashion like the Icinga 1.x distributed model. If you
-> are looking for a real distributed architecture with Icinga 2, scroll down.
-
## <a id="agent-based-checks-snmp-traps"></a> Passive Check Results and SNMP Traps
and specific trap handlers passing the check results to Icinga 2.
Following the SNMPTT [Format](http://snmptt.sourceforge.net/docs/snmptt.shtml#SNMPTT.CONF-FORMAT)
-documentation and the Icinga external command syntax found [here](24-appendix.md#external-commands-list-detail)
+documentation and the Icinga external command syntax found [here](23-appendix.md#external-commands-list-detail)
we can create generic services that can accommodate any number of hosts for a given scenario.
### <a id="simple-traps"></a> Simple SNMP Traps
vars.dummy_state = 2
vars.dummy_text = "No passive check result received."
}
+
### <a id="scheduling-downtime"></a> Scheduling a downtime
-This can either happen through a web interface or by sending an [external command](15-features.md#external-commands)
+This can either happen through a web interface or by sending an [external command](14-features.md#external-commands)
to the external command pipe provided by the `ExternalCommandListener` configuration.
Fixed downtimes require a start and end time (a duration will be ignored).
### <a id="recurring-downtimes"></a> Recurring Downtimes
-[ScheduledDowntime objects](6-object-types.md#objecttype-scheduleddowntime) can be used to set up
+[ScheduledDowntime objects](9-object-types.md#objecttype-scheduleddowntime) can be used to set up
recurring downtimes for services.
Example:
## <a id="timeperiods"></a> Time Periods
-[Time Periods](6-object-types.md#objecttype-timeperiod) define
+[Time Periods](9-object-types.md#objecttype-timeperiod) define
time ranges in Icinga where event actions are triggered, for
example whether a service check is executed or not within
the `check_period` attribute. Or a notification should be sent to
want to send out any notification during the holiday season,
or if you only want to allow small time windows for executed checks.
-The [TimePeriod object](6-object-types.md#objecttype-timeperiod)
+The [TimePeriod object](9-object-types.md#objecttype-timeperiod)
provides the `includes` and `excludes` attributes to solve this issue.
`prefer_includes` defines whether included or excluded time periods are
preferred.
There is a limited scope where functions can be used as object attributes such as:
* As value for [Custom Attributes](3-monitoring-basics.md#custom-attributes-functions)
-* Returning boolean expressions for [set_if](5-advanced-topics.md#use-functions-command-arguments-setif) inside command arguments
-* Returning a [command](5-advanced-topics.md#use-functions-command-attribute) array inside command objects
+* Returning boolean expressions for [set_if](8-advanced-topics.md#use-functions-command-arguments-setif) inside command arguments
+* Returning a [command](8-advanced-topics.md#use-functions-command-attribute) array inside command objects
The other way around you can create objects dynamically using your own global functions.
>
> Functions called inside command objects share the same global scope as runtime macros.
> Therefore you can access host custom attributes like `host.vars.os`, or any other
-> object attribute from inside the function definition used for [set_if](5-advanced-topics.md#use-functions-command-arguments-setif) or [command](5-advanced-topics.md#use-functions-command-attribute).
+> object attribute from inside the function definition used for [set_if](8-advanced-topics.md#use-functions-command-arguments-setif) or [command](8-advanced-topics.md#use-functions-command-attribute).
Tips when implementing functions:
-* Use [log()](19-library-reference.md#global-functions) to dump variables. You can see the output
+* Use [log()](18-library-reference.md#global-functions) to dump variables. You can see the output
inside the `icinga2.log` file depending in your log severity
* Use the `icinga2 console` to test basic functionality (e.g. iterating over a dictionary)
* Build them step-by-step. You can always refactor your code later on.
### <a id="use-functions-command-arguments-setif"></a> Use Functions in Command Arguments set_if
The `set_if` attribute inside the command arguments definition in the
-[CheckCommand object definition](6-object-types.md#objecttype-checkcommand) is primarily used to
+[CheckCommand object definition](9-object-types.md#objecttype-checkcommand) is primarily used to
evaluate whether the command parameter should be set or not.
By default you can evaluate runtime macros for their existence. If the result is not an empty
The more significant problem was to only add the command parameter `--disk` to the plugin call
when the dictionary `compellent` contains the key `disks`, and omit it if not found.
-By defining `set_if` as [abbreviated lambda function](18-language-reference.md#nullary-lambdas)
+By defining `set_if` as [abbreviated lambda function](17-language-reference.md#nullary-lambdas)
and evaluating the host custom attribute `compellent` containing the `disks` this problem was
solved like this:
}
}
-This implementation uses the dictionary type method [contains](19-library-reference.md#dictionary-contains)
+This implementation uses the dictionary type method [contains](18-library-reference.md#dictionary-contains)
and will fail if `host.vars.compellent` is not of the type `Dictionary`.
-Therefore you can extend the checks using the [typeof](18-language-reference.md#types) function.
+Therefore you can extend the checks using the [typeof](17-language-reference.md#types) function.
You can test the types using the `icinga2 console`:
### <a id="use-functions-command-attribute"></a> Use Functions as Command Attribute
-This comes in handy for [NotificationCommands](6-object-types.md#objecttype-notificationcommand)
-or [EventCommands](6-object-types.md#objecttype-eventcommand) which does not require
+This comes in handy for [NotificationCommands](9-object-types.md#objecttype-notificationcommand)
+or [EventCommands](9-object-types.md#objecttype-eventcommand) which does not require
a returned checkresult including state/output.
The following example was taken from the community support channels. The requirement was to
If a simple expression for matching a name or checking if an item
exists in an array or dictionary does not fit, you should consider
-writing your own global [functions](18-language-reference.md#functions).
+writing your own global [functions](17-language-reference.md#functions).
You can call them inside `assign where` and `ignore where` expressions
for [apply rules](3-monitoring-basics.md#using-apply-expressions) or
[group assignments](3-monitoring-basics.md#group-assign-intro) just like
-any other global functions for example [match](19-library-reference.md#global-functions).
+any other global functions for example [match](18-library-reference.md#global-functions).
The following example requires the host `myprinter` being added
to the host group `printers-lexmark` but only if the host uses
## <a id="access-object-attributes-at-runtime"></a> Access Object Attributes at Runtime
-The [Object Accessor Functions](19-library-reference.md#object-accessor-functions)
+The [Object Accessor Functions](18-library-reference.md#object-accessor-functions)
can be used to retrieve references to other objects by name.
This allows you to access configuration and runtime object attributes. A detailed
-list can be found [here](6-object-types.md#object-types).
+list can be found [here](9-object-types.md#object-types).
Simple cluster example for accessing two host object states and calculating a virtual
cluster state and output:
Config objects share these runtime attributes which cannot be
modified by the user. You can access these attributes using
-the [Icinga 2 API](9-icinga2-api.md#icinga2-api-config-objects).
+the [Icinga 2 API](12-icinga2-api.md#icinga2-api-config-objects).
Name |Description
--------------------------|--------------------------
type | Object type.
original_attributes | Original values of object attributes modified at runtime.
active | Object is active (e.g. a service being checked).
- paused | Object has been paused at runtime (e.g. [IdoMysqlConnection](6-object-types.md#objecttype-idomysqlconnection). Defaults to `false`.
+ paused | Object has been paused at runtime (e.g. [IdoMysqlConnection](9-object-types.md#objecttype-idomysqlconnection). Defaults to `false`.
templates | Templates imported on object compilation.
- package | [Configuration package name](9-icinga2-api.md#icinga2-api-config-management) this object belongs to. Local configuration is set to `_etc`, runtime created objects use `_api`.
+ package | [Configuration package name](12-icinga2-api.md#icinga2-api-config-management) this object belongs to. Local configuration is set to `_etc`, runtime created objects use `_api`.
## <a id="objecttype-apilistener"></a> ApiListener
client\_cn |**Optional.** Client Common Name (CN).
permissions |**Required.** Array of permissions. Either as string or dictionary with the keys `permission` and `filter`. The latter must be specified as function.
-Available permissions are described in the [API permissions](9-icinga2-api.md#icinga2-api-permissions)
+Available permissions are described in the [API permissions](12-icinga2-api.md#icinga2-api-permissions)
chapter.
## <a id="objecttype-checkcommand"></a> CheckCommand
end_time | **Required.** The end time as unix timestamp.
duration | **Required.** The duration as number.
entry_time | **Optional.** The unix timestamp when this downtime was added.
- fixed | **Optional.** Whether the downtime is fixed (true) or flexible (false). Defaults to flexible. Details in the [advanced topics chapter](5-advanced-topics.md#fixed-flexible-downtimes).
+ fixed | **Optional.** Whether the downtime is fixed (true) or flexible (false). Defaults to flexible. Details in the [advanced topics chapter](8-advanced-topics.md#fixed-flexible-downtimes).
triggers | **Optional.** List of downtimes which should be triggered by this downtime.
Runtime Attributes:
timeout |**Optional.** The command timeout in seconds. Defaults to 60 seconds.
arguments |**Optional.** A dictionary of command arguments.
-Command arguments can be used the same way as for [CheckCommand objects](6-object-types.md#objecttype-checkcommand-arguments).
+Command arguments can be used the same way as for [CheckCommand objects](9-object-types.md#objecttype-checkcommand-arguments).
## <a id="objecttype-externalcommandlistener"></a> ExternalCommandListener
enable_send_metadata | **Optional.** Send additional metadata metrics. Defaults to `false`.
enable_legacy_mode | **Optional.** Enable legacy mode for schema < 2.4. **Note**: This will be removed in future versions.
-Additional usage examples can be found [here](15-features.md#graphite-carbon-cache-writer).
+Additional usage examples can be found [here](14-features.md#graphite-carbon-cache-writer).
> **Best Practice**
>
-> Assign host group members using the [group assign](18-language-reference.md#group-assign) rules.
+> Assign host group members using the [group assign](17-language-reference.md#group-assign) rules.
Example:
table\_prefix |**Optional.** MySQL database table prefix. Defaults to "icinga\_".
instance\_name |**Optional.** Unique identifier for the local Icinga 2 instance. Defaults to "default".
instance\_description|**Optional.** Description for the Icinga 2 instance.
- enable_ha |**Optional.** Enable the high availability functionality. Only valid in a [cluster setup](13-distributed-monitoring-ha.md#high-availability-db-ido). Defaults to "true".
- failover_timeout | **Optional.** Set the failover timeout in a [HA cluster](13-distributed-monitoring-ha.md#high-availability-db-ido). Must not be lower than 60s. Defaults to "60s".
+ enable_ha |**Optional.** Enable the high availability functionality. Only valid in a [cluster setup](6-distributed-monitoring.md#distributed-monitoring-high-availability-db-ido). Defaults to "true".
+ failover_timeout | **Optional.** Set the failover timeout in a [HA cluster](6-distributed-monitoring.md#distributed-monitoring-high-availability-db-ido). Must not be lower than 60s. Defaults to "60s".
cleanup |**Optional.** Dictionary with items for historical table cleanup.
categories |**Optional.** Array of information types that should be written to the database.
table\_prefix |**Optional.** PostgreSQL database table prefix. Defaults to "icinga\_".
instance\_name |**Optional.** Unique identifier for the local Icinga 2 instance. Defaults to "default".
instance\_description|**Optional.** Description for the Icinga 2 instance.
- enable_ha |**Optional.** Enable the high availability functionality. Only valid in a [cluster setup](13-distributed-monitoring-ha.md#high-availability-db-ido). Defaults to "true".
- failover_timeout | **Optional.** Set the failover timeout in a [HA cluster](13-distributed-monitoring-ha.md#high-availability-db-ido). Must not be lower than 60s. Defaults to "60s".
+ enable_ha |**Optional.** Enable the high availability functionality. Only valid in a [cluster setup](6-distributed-monitoring.md#distributed-monitoring-high-availability-db-ido). Defaults to "true".
+ failover_timeout | **Optional.** Set the failover timeout in a [HA cluster](6-distributed-monitoring.md#distributed-monitoring-high-availability-db-ido). Must not be lower than 60s. Defaults to "60s".
cleanup |**Optional.** Dictionary with items for historical table cleanup.
categories |**Optional.** Array of information types that should be written to the database.
## <a id="objecttype-livestatuslistener"></a> LiveStatusListener
Livestatus API interface available as TCP or UNIX socket. Historical table queries
-require the [CompatLogger](6-object-types.md#objecttype-compatlogger) feature enabled
+require the [CompatLogger](9-object-types.md#objecttype-compatlogger) feature enabled
pointing to the log files using the `compat_log_path` configuration attribute.
Example:
timeout |**Optional.** The command timeout in seconds. Defaults to 60 seconds.
arguments |**Optional.** A dictionary of command arguments.
-Command arguments can be used the same way as for [CheckCommand objects](6-object-types.md#objecttype-checkcommand-arguments).
+Command arguments can be used the same way as for [CheckCommand objects](9-object-types.md#objecttype-checkcommand-arguments).
## <a id="objecttype-notificationcomponent"></a> NotificationComponent
Name |Description
----------------|----------------
- enable\_ha |**Optional.** Enable the high availability functionality. Only valid in a [cluster setup](13-distributed-monitoring-ha.md#high-availability-notifications). Disabling this currently only affects reminder notifications. Defaults to "true".
+ enable\_ha |**Optional.** Enable the high availability functionality. Only valid in a [cluster setup](6-distributed-monitoring.md#distributed-monitoring-high-availability-notifications). Disabling this currently only affects reminder notifications. Defaults to "true".
## <a id="objecttype-opentsdbwriter"></a> OpenTsdbWriter
> to just create a `ScheduledDowntime` template and use the `apply` keyword to assign the
> scheduled downtime to a number of hosts or services. Use the `to` keyword to set the specific target
> type for `Host` or `Service`.
-> Check the [recurring downtimes](5-advanced-topics.md#recurring-downtimes) example for details.
+> Check the [recurring downtimes](8-advanced-topics.md#recurring-downtimes) example for details.
Example:
> **Best Practice**
>
-> Assign service group members using the [group assign](18-language-reference.md#group-assign) rules.
+> Assign service group members using the [group assign](17-language-reference.md#group-assign) rules.
Example:
> **Best Practice**
>
-> Assign user group members using the [group assign](18-language-reference.md#group-assign) rules.
+> Assign user group members using the [group assign](17-language-reference.md#group-assign) rules.
Example:
# <a id="value-types"></a> Value Types
-In addition to [configuration objects](6-object-types.md) Icinga 2 also uses a few other types to represent its internal state. The following types are exposed via the [API](9-icinga2-api.md#icinga2-api).
+In addition to [configuration objects](9-object-types.md#object-types) Icinga 2 also uses a few other types to represent its internal state. The following types are exposed via the [API](12-icinga2-api.md#icinga2-api).
## <a id="value-types-checkresult"></a> CheckResult
--------------------------|---------------|-----------------
exit_status | Number | The exit status returned by the check execution.
output | String | The check output.
- performance_data | Array | Array of [performance data values](6-object-types.md#value-types-perfdatavalue).
+ performance_data | Array | Array of [performance data values](9-object-types.md#value-types-perfdatavalue).
check_source | String | Name of the node executing the check.
state | Number | The current state (0 = OK, 1 = WARNING, 2 = CRITICAL, 3 = UNKNOWN).
command | Value | Array of command with shell-escaped arguments or command line string.
## <a id="value-types-perfdatavalue"></a> PerfdataValue
-Icinga 2 parses performance data strings returned by check plugins and makes the information available to external interfaces (e.g. [GraphiteWriter](6-object-types.md#objecttype-graphitewriter) or the [Icinga 2 API](9-icinga2-api.md#icinga2-api)).
+Icinga 2 parses performance data strings returned by check plugins and makes the information available to external interfaces (e.g. [GraphiteWriter](9-object-types.md#objecttype-graphitewriter) or the [Icinga 2 API](12-icinga2-api.md#icinga2-api)).
Name | Type | Description
--------------------------|---------------|-----------------
label | String | Performance data label.
value | Number | Normalized performance data value without unit.
counter | Boolean | Enabled if the original value contains `c` as unit. Defaults to `false`.
- unit | String | Unit of measurement (`seconds`, `bytes`. `percent`) according to the [plugin API](14-addons-plugins.md#plugin-api).
+ unit | String | Unit of measurement (`seconds`, `bytes`. `percent`) according to the [plugin API](5-service-monitoring.md#service-monitoring-plugin-api).
crit | Value | Critical threshold value.
warn | Value | Warning threshold value.
min | Value | Minimum value returned by the check.
id = match.group("id")
if id in anchors:
- print "Anchor '%s' is used multiple times: in %s and %s" % (id, file, anchors[id])
+ print "Error: Anchor '%s' is used multiple times: in %s and %s" % (id, file, anchors[id])
anchors[match.group("id")] = file
try:
file = os.path.basename(anchors[id])
except KeyError:
- print "Unmatched anchor: %s" % (id)
+ print "Error: Unmatched anchor: %s" % (id)
file = ""
return "[%s](%s#%s)" % (match.group("text"), file, id)
for file in sys.argv[1:]:
text = open(file).read()
+ print "> Processing file '%s'..." % (file)
new_text = re.sub(r"\[(?P<text>.*?)\]\((?P<file>[0-9-a-z\.]+)?#(?P<id>[^#\)]+)\)", update_anchor, text)
open(file, "w").write(new_text)
- [2-getting-started.md, Getting Started]
- [3-monitoring-basics.md, Monitoring Basics]
- [4-configuring-icinga-2.md, Configuring Icinga 2]
-- [5-advanced-topics.md, Advanced Topics]
-- [6-object-types.md, Object Types]
-- [7-icinga-template-library.md, Icinga Template Library]
-- [8-cli-commands.md, CLI Commands]
-- [9-icinga2-api.md, Icinga 2 API]
-- [10-monitoring-remote-systems.md, Monitoring Remote Systems]
-- [11-icinga2-client.md, Icinga 2 Client]
-- [12-agent-based-checks.md, Additional Agent-based Checks]
-- [13-distributed-monitoring-ha.md, Distributed Monitoring and High Availability]
-- [14-addons-plugins.md, Addons and Plugins]
-- [15-features.md, Features]
-- [16-troubleshooting.md, Troubleshooting]
-- [17-upgrading-icinga-2.md, Upgrading Icinga 2]
-- [18-language-reference.md, Language Reference]
-- [19-library-reference.md, Library Reference]
-- [20-script-debugger.md, Script Debugger]
-- [21-development.md, Development]
-- [22-selinux.md, SELinux]
-- [23-migrating-from-icinga-1x.md, Migrating from Icinga 1.x]
-- [24-appendix.md, Appendix]
+- [5-service-monitoring.md, Service Monitoring]
+- [6-distributed-monitoring.md, Distributed Monitoring]
+- [7-agent-based-monitoring.md, Agent Based Monitoring]
+- [8-advanced-topics.md, Advanced Topics]
+- [9-object-types.md, Object Types]
+- [10-icinga-template-library.md, Icinga Template Library]
+- [11-cli-commands.md, CLI Commands]
+- [12-icinga2-api.md, Icinga 2 API]
+- [13-addons.md, Addons]
+- [14-features.md, Features]
+- [15-troubleshooting.md, Troubleshooting]
+- [16-upgrading-icinga-2.md, Upgrading Icinga 2]
+- [17-language-reference.md, Language Reference]
+- [18-library-reference.md, Library Reference]
+- [19-script-debugger.md, Script Debugger]
+- [20-development.md, Development]
+- [21-selinux.md, SELinux]
+- [22-migrating-from-icinga-1x.md, Migrating from Icinga 1.x]
+- [23-appendix.md, Appendix]
theme: readthedocs
markdown_extensions: [smarty]
extra_javascript: [scroll.js]
projects / icinga2 / commitdiff
