1 # Upgrading Icinga 2 <a id="upgrading-icinga-2"></a>
3 Upgrading Icinga 2 is usually quite straightforward. Ordinarily the only manual steps involved
4 are scheme updates for the IDO database.
6 Specific version upgrades are described below. Please note that version
7 updates are incremental. An upgrade from v2.6 to v2.8 requires to
8 follow the instructions for v2.7 too.
10 ## Upgrading to v2.10 <a id="upgrading-to-2-10"></a>
12 ### Path Constant Changes <a id="upgrading-to-2-10-path-constant-changes"></a>
14 During package upgrades you may see a notice that the configuration
15 content of features has changed. This is due to a more general approach
16 with path constants in v2.10.
18 The known constants `SysconfDir` and `LocalStateDir` stay intact and won't
20 If you are using these constants in your own custom command definitions
21 or other objects, you are advised to revise them and update them according
22 to the [documentation](17-language-reference.md#icinga-constants).
27 object NotificationCommand "mail-service-notification" {
28 - command = [ SysconfDir + "/icinga2/scripts/mail-service-notification.sh" ]
29 + command = [ ConfigDir + "/scripts/mail-service-notification.sh" ]
32 If you have the `ICINGA2_RUN_DIR` environment variable configured in the
33 sysconfig file, you need to rename it to `ICINGA2_INIT_RUN_DIR`. `ICINGA2_STATE_DIR`
34 has been removed and this setting has no effect.
38 > This is important if you rely on the sysconfig configuration in your own scripts.
40 ### New Constants <a id="upgrading-to-2-10-path-new-constants"></a>
42 New [Icinga constants](17-language-reference.md#icinga-constants) have been added in this release.
44 * `Environment` for specifying the Icinga environment. Defaults to not set.
45 * `ApiBindHost` and `ApiBindPort` to allow overriding the default ApiListener values. This will be used for an Icinga addon only.
47 ### Configuration: Namespaces <a id="upgrading-to-2-10-configuration-namespaces"></a>
49 The keywords `namespace` and `using` are now [reserved](17-language-reference.md#reserved-keywords) for the namespace functionality provided
50 with v2.10. Read more about how it works [here](17-language-reference.md#namespaces).
52 ### Configuration: ApiListener <a id="upgrading-to-2-10-configuration-apilistener"></a>
54 Anonymous JSON-RPC connections in the cluster can now be configured with `max_anonymous_clients`
56 The corresponding REST API results from `/v1/status/ApiListener` in `json_rpc` have been renamed
57 from `clients` to `anonymous_clients` to better reflect their purpose. Authenticated clients
58 are counted as connected endpoints. A similar change is there for the performance data metrics.
60 The TLS handshake timeout defaults to 10 seconds since v2.8.2. This can now be configured
61 with the configuration attribute `tls_handshake_timeout`. Beware of performance issues
62 with setting a too high value.
64 ### API: schedule-downtime Action <a id="upgrading-to-2-10-api-schedule-downtime-action"></a>
66 The attribute `child_options` was previously accepting 0,1,2 for specific child downtime settings.
67 This behaviour stays intact, but the new proposed way are specific constants as values (`DowntimeNoChildren`, `DowntimeTriggeredChildren`, `DowntimeNonTriggeredChildren`).
69 ### Notifications: Recovery and Acknowledgement <a id="upgrading-to-2-10-notifications"></a>
71 When a user should be notified on `Problem` and `Acknowledgement`, v2.10 now checks during
72 the `Acknowledgement` notification event whether this user has been notified about a problem before.
75 types = [ Problem, Acknowledgement, Recovery ]
78 If **no** `Problem` notification was sent, and the types filter includes problems for this user,
79 the `Acknowledgement` notification is **not** sent.
81 In contrast to that, the following configuration always sends `Acknowledgement` notifications.
84 types = [ Acknowledgement, Recovery ]
87 This change also restores the old behaviour for `Recovery` notifications. The above configuration
88 leaving out the `Problem` type can be used to only receive recovery notifications. If `Problem`
89 is added to the types again, Icinga 2 checks whether it has notified a user of a problem when
90 sending the recovery notification.
92 More details can be found in [this PR](https://github.com/Icinga/icinga2/pull/6527).
94 ### Stricter configuration validation
96 Some config errors are now fatal. While it never worked before, icinga2 refuses to start now!
98 For example the following started to give a fatal error in 2.10:
102 endpoints = [ "master-server" ]
103 parent = "global-templates"
107 ```critical/config: Error: Zone 'XXX' can not have a global zone as parent.```
109 ### Package Changes <a id="upgrading-to-2-10-package-changes"></a>
111 Debian/Ubuntu drops the `libicinga2` package. `apt-get upgrade icinga2`
112 won't remove such packages leaving the upgrade in an unsatisfied state.
114 Please use `apt-get full-upgrade` or `apt-get dist-upgrade` instead, as explained [here](https://github.com/Icinga/icinga2/issues/6695#issuecomment-430585915).
116 On RHEL/CentOS/Fedora, `icinga2-libs` has been obsoleted. Unfortunately yum's dependency
117 resolver doesn't allow to install older versions than 2.10 then. Please
118 read [here](https://github.com/Icinga/icinga-packaging/issues/114#issuecomment-429264827)
121 RPM packages dropped the [Classic UI](16-upgrading-icinga-2.md#upgrading-to-2-8-removed-classicui-config-package)
122 package in v2.8, Debian/Ubuntu packages were forgotten. This is now the case with this
123 release. Icinga 1.x is EOL by the end of 2018, plan your migration to [Icinga Web 2](https://icinga.com/docs/icingaweb2/latest/).
125 ## Upgrading to v2.9 <a id="upgrading-to-2-9"></a>
127 ### Deprecation and Removal Notes <a id="upgrading-to-2-9-deprecation-removal-notes"></a>
129 - Deprecation of 1.x compatibility features: `StatusDataWriter`, `CompatLogger`, `CheckResultReader`. Their removal is scheduled for 2.11.
130 Icinga 1.x is EOL and will be out of support by the end of 2018.
131 - Removal of Icinga Studio. It always has been experimental and did not satisfy our high quality standards. We've therefore removed it.
133 ### Sysconfig Changes <a id="upgrading-to-2-9-sysconfig-changes"></a>
135 The security fixes in v2.8.2 required moving specific runtime settings
136 into the Sysconfig file and environment. This included that Icinga 2
137 would itself parse this file and read the required variables. This has generated
138 numerous false-positive log messages and led to many support questions. v2.9.0
139 changes this in the standard way to read these variables from the environment, and use
140 sane compile-time defaults.
144 > In order to upgrade, remove everything in the sysconfig file and re-apply
147 There is a bug with existing sysconfig files where path variables are not expanded
148 because Systemd [does not support](https://github.com/systemd/systemd/issues/2123)
149 shell variable expansion. This worked with SysVInit though.
151 Edit the sysconfig file and either remove everything, or edit this line
152 on RHEL 7. Modify the path for other distributions.
155 vim /etc/sysconfig/icinga2
157 -ICINGA2_PID_FILE=$ICINGA2_RUN_DIR/icinga2/icinga2.pid
158 +ICINGA2_PID_FILE=/run/icinga2/icinga2.pid
161 If you want to adjust the number of open files for the Icinga application
162 for example, you would just add this setting like this on RHEL 7:
165 vim /etc/sysconfig/icinga2
167 ICINGA2_RLIMIT_FILES=50000
170 Restart Icinga 2 afterwards, the Systemd service file automatically puts the
171 value into the application's environment where this is read on startup.
173 ### Setup Wizard Changes <a id="upgrading-to-2-9-setup-wizard-changes"></a>
175 Client and satellite setups previously had the example configuration in `conf.d` included
176 by default. This caused trouble on config sync, or with locally executed checks generating
177 wrong check results for command endpoint clients.
179 In v2.9.0 `node wizard`, `node setup` and the graphical Windows wizard will disable
180 the inclusion by default. You can opt-out and explicitly enable it again if needed.
182 In addition to the default global zones `global-templates` and `director-global`,
183 the setup wizards also offer to specify your own custom global zones and generate
184 the required configuration automatically.
186 The setup wizards also use full qualified names for Zone and Endpoint object generation,
187 either the default values (FQDN for clients) or the user supplied input. This removes
188 the dependency on the `NodeName` and `ZoneName` constant and helps to immediately see
189 the parent-child relationship. Those doing support will also see the benefit in production.
191 ### CLI Command Changes <a id="upgrading-to-2-9-cli-changes"></a>
193 The [node setup](06-distributed-monitoring.md#distributed-monitoring-automation-cli-node-setup)
194 parameter `--master_host` was deprecated and replaced with `--parent_host`.
195 This parameter is now optional to allow connection-less client setups similar to the `node wizard`
196 CLI command. The `parent_zone` parameter has been added to modify the parent zone name e.g.
197 for client-to-satellite setups.
199 The `api user` command which was released in v2.8.2 turned out to cause huge problems with
200 configuration validation, windows restarts and OpenSSL versions. It is therefore removed in 2.9,
201 the `password_hash` attribute for the ApiUser object stays intact but has no effect. This is to ensure
202 that clients don't break on upgrade. We will revise this feature in future development iterations.
204 ### Configuration Changes <a id="upgrading-to-2-9-config-changes"></a>
206 The CORS attributes `access_control_allow_credentials`, `access_control_allow_headers` and
207 `access_control_allow_methods` are now controlled by Icinga 2 and cannot be changed anymore.
209 ### Unique Generated Names <a id="upgrading-to-2-9-unique-name-changes"></a>
211 With the removal of RHEL 5 as supported platform, we can finally use real unique IDs.
212 This is reflected in generating names for e.g. API stage names. Previously it was a handcrafted
213 mix of local FQDN, timestamps and random numbers.
215 ### Custom Vars not updating <a id="upgrading-to-2-9-custom-vars-not-updating"></a>
217 A rare issue preventing the custom vars of objects created prior to 2.9.0 being updated when changed may occur. To
218 remedy this, truncate the customvar tables and restart Icinga 2. The following is an example of how to do this with mysql:
221 $ mysql -uroot -picinga icinga
222 MariaDB [icinga]> truncate icinga_customvariables;
223 Query OK, 0 rows affected (0.05 sec)
224 MariaDB [icinga]> truncate icinga_customvariablestatus;
225 Query OK, 0 rows affected (0.03 sec)
226 MariaDB [icinga]> exit
228 $ sudo systemctl restart icinga2
231 Custom vars should now stay up to date.
234 ## Upgrading to v2.8.2 <a id="upgrading-to-2-8-2"></a>
236 With version 2.8.2 the location of settings formerly found in `/etc/icinga2/init.conf` has changed. They are now
237 located in the sysconfig, `/etc/sysconfig/icinga2` (RPM) or `/etc/default/icinga2` (DPKG) on most systems. The
238 `init.conf` file has been removed and its settings will be ignored. These changes are only relevant if you edited the
239 `init.conf`. Below is a table displaying the new names for the affected settings.
241 Old `init.conf` | New `sysconfig/icinga2`
242 ----------------|------------------------
243 RunAsUser | ICINGA2\_USER
244 RunAsGroup | ICINGA2\_GROUP
245 RLimitFiles | ICINGA2\_RLIMIT\_FILES
246 RLimitProcesses | ICINGA2\_RLIMIT\_PROCESSES
247 RLimitStack | ICINGA2\_RLIMIT\_STACK
249 ## Upgrading to v2.8 <a id="upgrading-to-2-8"></a>
251 ### DB IDO Schema Update to 2.8.0 <a id="upgrading-to-2-8-db-ido"></a>
253 There are additional indexes and schema fixes which require an update.
255 Please proceed here for [MySQL](16-upgrading-icinga-2.md#upgrading-mysql-db) or [PostgreSQL](16-upgrading-icinga-2.md#upgrading-postgresql-db).
259 > `2.8.1.sql` fixes a unique constraint problem with fresh 2.8.0 installations.
260 > You don't need this update if you are upgrading from an older version.
262 ### Changed Certificate Paths <a id="upgrading-to-2-8-certificate-paths"></a>
264 The default certificate path was changed from `/etc/icinga2/pki` to
265 `/var/lib/icinga2/certs`.
268 ---------------------------------------------------|---------------------------------------------------
269 `/etc/icinga2/pki/icinga2-client1.localdomain.crt` | `/var/lib/icinga2/certs/icinga2-client1.localdomain.crt`
270 `/etc/icinga2/pki/icinga2-client1.localdomain.key` | `/var/lib/icinga2/certs/icinga2-client1.localdomain.key`
271 `/etc/icinga2/pki/ca.crt` | `/var/lib/icinga2/certs/ca.crt`
273 This applies to Windows clients in the same way: `%ProgramData%\etc\icinga2\pki`
274 was moved to `%ProgramData%\var\lib\icinga2\certs`.
277 ----------------------------------------------------------------|----------------------------------------------------------------
278 `%ProgramData%\etc\icinga2\pki\icinga2-client1.localdomain.crt` | `%ProgramData%\var\lib\icinga2\certs\icinga2-client1.localdomain.crt`
279 `%ProgramData%\etc\icinga2\pki\icinga2-client1.localdomain.key` | `%ProgramData%\var\lib\icinga2\certs\icinga2-client1.localdomain.key`
280 `%ProgramData%\etc\icinga2\pki\ca.crt` | `%ProgramData%\var\lib\icinga2\certs\ca.crt`
285 > The default expected path for client certificates is `/var/lib/icinga2/certs/ + NodeName + {.crt,.key}`.
286 > The `NodeName` constant is usually the FQDN and certificate common name (CN). Check the [conventions](06-distributed-monitoring.md#distributed-monitoring-conventions)
287 > section inside the Distributed Monitoring chapter.
289 The [setup CLI commands](06-distributed-monitoring.md#distributed-monitoring-setup-master) and the
290 default [ApiListener configuration](06-distributed-monitoring.md#distributed-monitoring-apilistener)
291 have been adjusted to these paths too.
293 The [ApiListener](09-object-types.md#objecttype-apilistener) object attributes `cert_path`, `key_path`
294 and `ca_path` have been deprecated and removed from the example configuration.
296 #### Migration Path <a id="upgrading-to-2-8-certificate-paths-migration-path"></a>
300 > Icinga 2 automatically migrates the certificates to the new default location if they
301 > are configured and detected in `/etc/icinga2/pki`.
303 During startup, the migration kicks in and ensures to copy the certificates to the new
304 location. This will also happen if someone updates the certificate files in `/etc/icinga2/pki`
305 to ensure that the new certificate location always has the latest files.
307 This has been implemented in the Icinga 2 binary to ensure it works on both Linux/Unix
308 and the Windows platform.
310 If you are not using the built-in CLI commands and setup wizards to deploy the client certificates,
311 please ensure to update your deployment tools/scripts. This mainly affects
317 * Custom scripts, e.g. Windows Powershell or self-made implementations
319 In order to support a smooth migration between versions older than 2.8 and future releases,
320 the built-in certificate migration path is planned to exist as long as the deprecated
321 `ApiListener` object attributes exist.
323 You are safe to use the existing configuration paths inside the `api` feature.
327 Look at the following example taken from the Director Linux deployment script for clients.
329 * Ensure that the default certificate path is changed from `/etc/icinga2/pki` to `/var/lib/icinga2/certs`.
332 -ICINGA2_SSL_DIR="${ICINGA2_CONF_DIR}/pki"
333 +ICINGA2_SSL_DIR="${ICINGA2_STATE_DIR}/lib/icinga2/certs"
336 * Remove the ApiListener configuration attributes.
339 object ApiListener "api" {
340 - cert_path = SysconfDir + "/icinga2/pki/${ICINGA2_NODENAME}.crt"
341 - key_path = SysconfDir + "/icinga2/pki/${ICINGA2_NODENAME}.key"
342 - ca_path = SysconfDir + "/icinga2/pki/ca.crt"
343 accept_commands = true
348 Test the script with a fresh client installation before putting it into production.
352 > Please support module and script developers in their migration. If you find
353 > any project which would require these changes, create an issue or a patchset in a PR
354 > and help them out. Thanks in advance!
356 ### On-Demand Signing and CA Proxy <a id="upgrading-to-2-8-on-demand-signing-ca-proxy"></a>
358 Icinga 2 v2.8 supports the following features inside the cluster:
360 * Forward signing requests from clients through a satellite instance to a signing master ("CA Proxy").
361 * Signing requests without a ticket. The master instance allows to list and sign CSRs ("On-Demand Signing").
363 In order to use these features, **all instances must be upgraded to v2.8**.
365 More details in [this chapter](06-distributed-monitoring.md#distributed-monitoring-setup-sign-certificates-master).
367 ### Windows Client <a id="upgrading-to-2-8-windows-client"></a>
369 Windows versions older than Windows 10/Server 2016 require the [Universal C Runtime for Windows](https://support.microsoft.com/en-us/help/2999226/update-for-universal-c-runtime-in-windows).
371 ### Removed Bottom Up Client Mode <a id="upgrading-to-2-8-removed-bottom-up-client-mode"></a>
373 This client mode was deprecated in 2.6 and was removed in 2.8.
375 The node CLI command does not provide `list` or `update-config` anymore.
379 > The old migration guide can be found on [GitHub](https://github.com/Icinga/icinga2/blob/v2.7.0/doc/06-distributed-monitoring.md#bottom-up-migration-to-top-down-).
381 The clients don't need to have a local `conf.d` directory included.
383 Icinga 2 continues to run with the generated and imported configuration.
384 You are advised to [migrate](https://github.com/Icinga/icinga2/issues/4798)
385 any existing configuration to the "top down" mode with the help of the
386 Icinga Director or config management tools such as Puppet, Ansible, etc.
389 ### Removed Classic UI Config Package <a id="upgrading-to-2-8-removed-classicui-config-package"></a>
391 The config meta package `classicui-config` and the configuration files
392 have been removed. You need to manually configure
393 this legacy interface. Create a backup of the configuration
394 before upgrading and re-configure it afterwards.
397 ### Flapping Configuration <a id="upgrading-to-2-8-flapping-configuration"></a>
399 Icinga 2 v2.8 implements a new flapping detection algorithm which splits the
400 threshold configuration into low and high settings.
402 `flapping_threshold` is deprecated and does not have any effect when flapping
403 is enabled. Please remove `flapping_threshold` from your configuration. This
404 attribute will be removed in v2.9.
406 Instead you need to use the `flapping_threshold_low` and `flapping_threshold_high`
407 attributes. More details can be found [here](08-advanced-topics.md#check-flapping).
409 ### Deprecated Configuration Attributes <a id="upgrading-to-2-8-deprecated-configuration"></a>
412 --------------|------------------
413 ApiListener | cert\_path (migration happens)
414 ApiListener | key\_path (migration happens)
415 ApiListener | ca\_path (migration happens)
416 Host, Service | flapping\_threshold (has no effect)
418 ## Upgrading to v2.7 <a id="upgrading-to-2-7"></a>
420 v2.7.0 provided new notification scripts and commands. Please ensure to
421 update your configuration accordingly. An advisory has been published [here](https://icinga.com/2017/08/23/advisory-for-icinga-2-v2-7-update-and-mail-notification-scripts/).
423 In case are having troubles with OpenSSL 1.1.0 and the
424 public CA certificates, please read [this advisory](https://icinga.com/2017/08/30/advisory-for-ssl-problems-with-leading-zeros-on-openssl-1-1-0/)
425 and check the [troubleshooting chapter](15-troubleshooting.md#troubleshooting).
427 If Icinga 2 fails to start with an empty reference to `$ICINGA2_CACHE_DIR`
428 ensure to set it inside `/etc/sysconfig/icinga2` (RHEL) or `/etc/default/icinga2` (Debian).
430 RPM packages will put a file called `/etc/sysconfig/icinga2.rpmnew`
431 if you have modified the original file.
436 vim /etc/sysconfig/icinga2
438 ICINGA2_CACHE_DIR=/var/cache/icinga2
440 systemctl restart icinga2
443 ## Upgrading the MySQL database <a id="upgrading-mysql-db"></a>
445 If you want to upgrade an existing Icinga 2 instance, check the
446 `/usr/share/icinga2-ido-mysql/schema/upgrade` directory for incremental schema upgrade file(s).
450 > If there isn't an upgrade file for your current version available, there's nothing to do.
452 Apply all database schema upgrade files incrementally.
455 # mysql -u root -p icinga < /usr/share/icinga2-ido-mysql/schema/upgrade/<version>.sql
458 The Icinga 2 DB IDO feature checks the required database schema version on startup
459 and generates an log message if not satisfied.
462 **Example:** You are upgrading Icinga 2 from version `2.4.0` to `2.8.0`. Look into
463 the `upgrade` directory:
466 $ ls /usr/share/icinga2-ido-mysql/schema/upgrade/
467 2.0.2.sql 2.1.0.sql 2.2.0.sql 2.3.0.sql 2.4.0.sql 2.5.0.sql 2.6.0.sql 2.8.0.sql
470 There are two new upgrade files called `2.5.0.sql`, `2.6.0.sql` and `2.8.0.sql`
471 which must be applied incrementally to your IDO database.
474 # mysql -u root -p icinga < /usr/share/icinga2-ido-mysql/schema/upgrade/2.5.0.sql
475 # mysql -u root -p icinga < /usr/share/icinga2-ido-mysql/schema/upgrade/2.6.0.sql
476 # mysql -u root -p icinga < /usr/share/icinga2-ido-mysql/schema/upgrade/2.8.0.sql
479 ## Upgrading the PostgreSQL database <a id="upgrading-postgresql-db"></a>
481 If you want to upgrade an existing Icinga 2 instance, check the
482 `/usr/share/icinga2-ido-pgsql/schema/upgrade` directory for incremental schema upgrade file(s).
486 > If there isn't an upgrade file for your current version available, there's nothing to do.
488 Apply all database schema upgrade files incrementally.
491 # export PGPASSWORD=icinga
492 # psql -U icinga -d icinga < /usr/share/icinga2-ido-pgsql/schema/upgrade/<version>.sql
495 The Icinga 2 DB IDO feature checks the required database schema version on startup
496 and generates an log message if not satisfied.
498 **Example:** You are upgrading Icinga 2 from version `2.4.0` to `2.8.0`. Look into
499 the `upgrade` directory:
502 $ ls /usr/share/icinga2-ido-pgsql/schema/upgrade/
503 2.0.2.sql 2.1.0.sql 2.2.0.sql 2.3.0.sql 2.4.0.sql 2.5.0.sql 2.6.0.sql 2.8.0.sql
506 There are two new upgrade files called `2.5.0.sql`, `2.6.0.sql` and `2.8.0.sql`
507 which must be applied incrementally to your IDO database.
510 # export PGPASSWORD=icinga
511 # psql -U icinga -d icinga < /usr/share/icinga2-ido-pgsql/schema/upgrade/2.5.0.sql
512 # psql -U icinga -d icinga < /usr/share/icinga2-ido-pgsql/schema/upgrade/2.6.0.sql
513 # psql -U icinga -d icinga < /usr/share/icinga2-ido-pgsql/schema/upgrade/2.8.0.sql