Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents


This documentation describes how to upgrade your current installation of openITCOCKPIT 3.7.3 (or later) to openITCOCKPIT 4.


Info

Before you start please create a backup of your system!

It is recommended to test your backup and the upgrade process within a test lab setup!

Requirements

  • Make sure your openITCOCKPIT installation is on the latest version.
  • openITCOCKPIT >= 3.7.3
  • All operating system updates are installed
  • A valid backup of openITCOCKPIT (in case something goes wrong)
  • At least 20GB of free disk space
  • Distributed Monitoring: It is also required to update all Satellite Systems to openITCOCKPIT 4
  • Run all commands as root user.

Supported distributions

Ubuntu Focal (20.04)Ubuntu Bionic (18.04)Ubuntu Xenial (16.04)Ubuntu Trusty (14.04)Debian Buster (10)Debian Stretch (9)Debian Jessie (8)
(tick)(tick)(tick)(warning)(error)(tick)(tick)(warning)(error)

(warning) Ubuntu Xenial (16.04)

openITCOCKPIT 4 has no support to run on Ubuntu Xenial. By upgrading to openITCOCKPIT 4 a distribution upgrade to Ubuntu Bionic (18.04) will be done automatically!

(warning) Debian Stretch (9)

openITCOCKPIT 4 has no support to run on Debian Stretch. By upgrading to openITCOCKPIT 4 a distribution upgrade to Debian Buster (10) will be done automatically!


Prepare System

If you are using Ubuntu Xenial you need to Remove the lxd Package which may cause errors during the upgrade to Ubuntu Bionic and openITCOCKPIT v4.

Code Block
titleUbuntu Xenial
#only run on Ubuntu Xenial!
apt-get remove lxd


Step 1 - Update operating system

Code Block
apt-get update
apt-get dist-upgrade


Step 2 - Create Database Backup

Note

Make sure to check that your system has enough free disk space!

Code Block
mysqldump --defaults-extra-file=/etc/mysql/debian.cnf --databases openitcockpit --flush-privileges --single-transaction --triggers --routines --events --hex-blob \
  --ignore-table=openitcockpit.nagios_externalcommands \
  --ignore-table=openitcockpit.nagios_hostchecks \
  --ignore-table=openitcockpit.nagios_logentries \
  --ignore-table=openitcockpit.nagios_servicechecks \
  --ignore-table=openitcockpit.nagios_systemcommands \
  --ignore-table=openitcockpit.nagios_timedeventqueue \
  --ignore-table=openitcockpit.nagios_timedevents \
  --ignore-table=openitcockpit.statusengine_dbversion \
  --ignore-table=openitcockpit.statusengine_host_acknowledgements \
  --ignore-table=openitcockpit.statusengine_host_downtimehistory \
  --ignore-table=openitcockpit.statusengine_host_notifications \
  --ignore-table=openitcockpit.statusengine_host_scheduleddowntimes \
  --ignore-table=openitcockpit.statusengine_host_statehistory \
  --ignore-table=openitcockpit.statusengine_hostchecks \
  --ignore-table=openitcockpit.statusengine_hoststatus \
  --ignore-table=openitcockpit.statusengine_logentries \
  --ignore-table=openitcockpit.statusengine_nodes \
  --ignore-table=openitcockpit.statusengine_perfdata \
  --ignore-table=openitcockpit.statusengine_service_acknowledgements \
  --ignore-table=openitcockpit.statusengine_service_downtimehistory \
  --ignore-table=openitcockpit.statusengine_service_notifications \
  --ignore-table=openitcockpit.statusengine_service_scheduleddowntimes \
  --ignore-table=openitcockpit.statusengine_service_statehistory \
  --ignore-table=openitcockpit.statusengine_servicechecks \
  --ignore-table=openitcockpit.statusengine_servicestatus \
  --ignore-table=openitcockpit.statusengine_tasks \
  --ignore-table=openitcockpit.statusengine_users \
  >/root/openitcockpit_dump_v3_upgrade.sql


Step 3 - Run openITCOCKPIT 4 compatibility check


To ensure that your system is able to get upgraded to openITCOCKPIT 4.x we provide a compatibility check script.

Run the script on your openITCOCKPIT instance.

Code Block
curl -s https://raw.githubusercontent.com/it-novum/openITCOCKPIT/development/bin/scripts/oitc3to4.sh | sudo /bin/bash

No changes had been made to your system yet!


Warning

Do not continue as long as you get any errors or warnings!

Resolve all reported errors and warning. Rerun the compatibility check to ensure that all issues were resolved.



Step 3.1 - Distributed monitoring

It is also required to upgrade all Satellite Systems to openITCOCKPIT 4.

Within the current release it is not possible to connect openITCOCKPIT 3.x satellite systems with an openITCOCKPIT 4.x master system. If you for some reason cannot upgrade your Satellite systems - please do not upgrade your Master system now!

Step 3.2 - Upgrade openITCOCKPIT

Note

Stable SSH connection required!

The upgrade process could take a while. To avoid SSH connectivity issues it is recommended to use tmux or screen.


The compatibility check will print the required commands to upgrade your installation of openITCOCKPIT depending on the installed modules and distribution. The upgrade commands get generated for each system individually.


Warning

DO NOT use any other commands to upgrade your system!



Ubuntu Users

Please make sure your system has enabled "universe" repositories

Code Block
add-apt-repository universe

Debian Users

Please make sure your system has enabled "non-free" repositories

Code Block
title/etc/apt/sources.list
deb http://ftp.de.debian.org/debian/ buster main contrib non-free
deb http://ftp.de.debian.org/debian/ buster-updates main contrib non-free
deb http://security.debian.org/ buster/updates main contrib non-free


Run the commands provided by the compatibility check script to upgrade your openITCOCKPIT.


During the upgrade you may get ask to overwrite some configuration files with newer versions. These are configuration files from the operating system itself so you can choose if you want to keep your old configuration or use the new one. This has no effect on the openITCOCKPIT upgrade.

Ubuntu Xenial and Debian Stretch users only

The upgrade will also upgrade your distribution to the next release. (Ubuntu Xenial to Bionic and Debian Strecht to Buster). If you have installed any custom software or loaded any third party repositories please make sure they are compatible with Ubuntu Bionic/Debian Buster.


Step 4 - Reset user passwords


After the upgrade process, you need to reset all passwords of local users. If you are using LDAP you don't need to do this.

Reset the password of one user

To reset the password of one singe user you could execute the following command:

Code Block
oitc reset_password --print


The System will generate a random password and send the new password to the users email address.

By passing the --print option the new password will also be printed to the terminal.

Passing the --no-mail option does not send the email with the new password to the user.




Reset all user passwords

To reset the password of all local users you could execute the following command:

Code Block
oitc update3_to4 --reset-all-passwords

The System will generate a random password for each user and send the new password to the users email address.


Step 5 - Clear Browser cache

Before you login to the new openITCOCKPIT interface you should clear your Browser cache.

Mozilla Firefox: https://support.mozilla.org/en-US/kb/how-clear-firefox-cache

Google Chrome: https://support.google.com/accounts/answer/32050?co=GENIE.Platform%3DDesktop&hl=en

Step 5.1 - Login to the openITCOCKPIT interface


Step 6 - Restart NSTA (Distributed Monitoring only)

To start processing of Satellite check results it is required that you start the NSTA daemon.

Code Block
systemctl restart nsta

Step 7 - Reboot your system

Especially when upgrading from Ubuntu Xenial or Debian Strech a reboot of the system is recommended.


Update openITCOCKPIT Satellite Systems (Distributed Monitoring only)

Install all system updates first

Code Block
apt-get update
apt-get dist-upgrade

Execute the openITCOCKPIT Satellite checklist script and follow the instructions

Code Block
curl -s https://raw.githubusercontent.com/it-novum/openITCOCKPIT/development/bin/scripts/oitcSat3to4.sh | sudo /bin/bash

It is recommended (but not required) to reboot the Satellite System after the upgrade is finished.

Known Issues

All distributions

The modules openitcockpit-module-discovery and openitcockpit-module-idoit are not available for openITCOCKPIT 4 yet. If you are using this modules please consider to not upgrade to openITCOCKPIT 4 now.

Cant login after password reset

If the Password was reset properly and the user still cant login with the message "Invalid username or password", the user must be manually set active in the Database. This can be done with the following SQL Command:

Code Block
UPDATE users SET is_active = 1 WHERE users.email= 'user@example.com';


Warnings and dependency problems during the upgrade process

During the upgrade you will properly see messages like

Code Block
Unpacking openitcockpit-module-distribute (4.0.4-20200721154853bionic) over (3.7.3-1ubuntu16.04) ...
dpkg: warning: unable to delete old directory '/etc/openitcockpit/app/Plugin/DistributeModule/Config/Schema': Directory not empty
dpkg: warning: unable to delete old directory '/etc/openitcockpit/app/Plugin/DistributeModule/Config': Directory not empty
dpkg: warning: unable to delete old directory '/etc/openitcockpit/app/Plugin/DistributeModule': Directory not empty
dpkg: warning: unable to delete old directory '/usr/share/openitcockpit/app/Plugin/DistributeModule': Directory not empty

or

Code Block
Removing php7.0-cli (7.0.33-0ubuntu0.16.04.15) ...
dpkg: php7.0-mysql: dependency problems, but removing anyway as you requested:
 phpmyadmin depends on php-mysql | php-mysqli | php-mysqlnd; however:
  Package php-mysql is not configured yet.
  Package php-mysqli is not installed.
  Package php7.2-mysql which provides php-mysqli is not configured yet.
  Package php7.0-mysql which provides php-mysqli is to be removed.
  Package php-mysqlnd is not installed.
  Package php7.2-mysql which provides php-mysqlnd is not configured yet.
  Package php7.0-mysql which provides php-mysqlnd is to be removed.
 phpmyadmin depends on php-mysql | php-mysqli | php-mysqlnd; however:
  Package php-mysql is not configured yet.
  Package php-mysqli is not installed.
  Package php7.2-mysql which provides php-mysqli is not configured yet.
  Package php7.0-mysql which provides php-mysqli is to be removed.
  Package php-mysqlnd is not installed.
  Package php7.2-mysql which provides php-mysqlnd is not configured yet.
  Package php7.0-mysql which provides php-mysqlnd is to be removed.

You can ignore these messages.

No Satellite check results on the Master Server

In the case that the Master server does not receive any check results from the Satellite systems a "Refresh of monitoring configuration" and a restart of the NSTA maybe resolve the issue.

Code Block
systemctl restart nsta

Ubuntu Focal (20.04)

openitcockpit-module-snmp-trap

Unfortunately the openitcockpit-module-snmp-trap module can not be installed on Ubuntu Focal at the moment due to Ubuntu related dependency issues.

Tip

This bug got resolved with the latest version of Ubuntu Focal.


https://bugs.launchpad.net/ubuntu/+source/net-snmp/+bug/1886658?comments=all


Troubleshooting

Network error

In case you had any connectivity issues you can try to repair the system with:

Code Block
apt-get install -f

You version of openITCOCKPIT is < 3.7.3!

The update process requires at least openITCOCKPIT 3.7.3. Please consider to check our blogposts (https://openitcockpit.io/blog/#blog) if you are upgrading from an older Version 3, especially when upgrading from Version 3.5 or older. Manual action is required if you want to have your performance graphs available in the newer versions.

Table nagios servicechecks is 36626 MB but /var/lib/mysql only has 16316 MB free disk space.

During the upgrade process all mysql tables get converted from UTF8 to UTF8mb4. This process requires a fair amount of disk space so please don't ignore these errors.

Upgrade from Ubuntu Xenial 16.04 to Bionic 18.04 and openITCOCKPIT v4 failed due to lxd package.

if the upgrade from Ubuntu Xenial fails due to the lxd package and this error:

Code Block
Old bridge configuration detected in /etc/default/lxd-bridge, upgrading
Unsetting deprecated profile options
Error: unknown command "profile" for "lxc"

Did you mean this?
	Profil

Run 'lxc --help' for usage.
dpkg: error processing package lxd (--configure):
 installed lxd package post-installation script subprocess returned error exit status 1

....

Fehler traten auf beim Bearbeiten von:
 lxd
E: Sub-process /usr/bin/dpkg returned an error code (1)

The frontend of openitcockpit is not visible in the browser

you need to execute the following commands:

Code Block
apt-get install -f
apt-get remove lxd
apt-get autoremove
reboot

#after reboot you need to trigger the UPGRADE script from itcockpit which was not triggered during the upgrade
/opt/openitc/frontend/bin/scripts/UPGRADE.sh

after that, the Frontend should be visible and the distribution upgrade has finished.

I still use Ubuntu Trusty or Debian Jessie

Please contact sales for commercial support.