Page tree
Skip to end of metadata
Go to start of metadata


This documentation describes how to upgrade your current installation of openITCOCKPIT 3.7.3 (or later) to openITCOCKPIT 4.


Before you start please create a backup of your system!

It is recommended to test your backup and the upgrade process within a test lab setup!

Requirements


  • Make sure your openITCOCKPIT installation is on the latest version.
  • openITCOCKPIT >= 3.7.3
  • All operating system updates are installed
  • A valid backup of openITCOCKPIT (in case something goes wrong)
  • At least 20GB of free disk space
  • Distributed Monitoring: It is also required to update all Satellite Systems to openITCOCKPIT 4
  • Run all commands as root user.

Supported distributions

Ubuntu Focal (20.04)Ubuntu Bionic (18.04)Ubuntu Xenial (16.04)Ubuntu Trusty (14.04)Debian Buster (10)Debian Stretch (9)Debian Jessie (8)
(tick)(tick)(tick)(warning)(error)(tick)(tick)(warning)(error)

(warning) Ubuntu Xenial (16.04)

openITCOCKPIT 4 has no support to run on Ubuntu Xenial. By upgrading to openITCOCKPIT 4 a distribution upgrade to Ubuntu Bionic (18.04) will be done automatically!

(warning) Debian Stretch (9)

openITCOCKPIT 4 has no support to run on Debian Stretch. By upgrading to openITCOCKPIT 4 a distribution upgrade to Debian Buster (10) will be done automatically!


Prepare System

If you are using Ubuntu Xenial you need to Remove the lxd Package which may cause errors during the upgrade to Ubuntu Bionic and openITCOCKPIT v4.

Ubuntu Xenial
#only run on Ubuntu Xenial!
apt-get remove lxd


Step 1 - Update operating system

apt-get update
apt-get dist-upgrade


Step 2 - Create Database Backup

Make sure to check that your system has enough free disk space!

mysqldump --defaults-extra-file=/etc/mysql/debian.cnf --databases openitcockpit --flush-privileges --single-transaction --triggers --routines --events --hex-blob \
  --ignore-table=openitcockpit.nagios_externalcommands \
  --ignore-table=openitcockpit.nagios_hostchecks \
  --ignore-table=openitcockpit.nagios_logentries \
  --ignore-table=openitcockpit.nagios_servicechecks \
  --ignore-table=openitcockpit.nagios_systemcommands \
  --ignore-table=openitcockpit.nagios_timedeventqueue \
  --ignore-table=openitcockpit.nagios_timedevents \
  --ignore-table=openitcockpit.statusengine_dbversion \
  --ignore-table=openitcockpit.statusengine_host_acknowledgements \
  --ignore-table=openitcockpit.statusengine_host_downtimehistory \
  --ignore-table=openitcockpit.statusengine_host_notifications \
  --ignore-table=openitcockpit.statusengine_host_scheduleddowntimes \
  --ignore-table=openitcockpit.statusengine_host_statehistory \
  --ignore-table=openitcockpit.statusengine_hostchecks \
  --ignore-table=openitcockpit.statusengine_hoststatus \
  --ignore-table=openitcockpit.statusengine_logentries \
  --ignore-table=openitcockpit.statusengine_nodes \
  --ignore-table=openitcockpit.statusengine_perfdata \
  --ignore-table=openitcockpit.statusengine_service_acknowledgements \
  --ignore-table=openitcockpit.statusengine_service_downtimehistory \
  --ignore-table=openitcockpit.statusengine_service_notifications \
  --ignore-table=openitcockpit.statusengine_service_scheduleddowntimes \
  --ignore-table=openitcockpit.statusengine_service_statehistory \
  --ignore-table=openitcockpit.statusengine_servicechecks \
  --ignore-table=openitcockpit.statusengine_servicestatus \
  --ignore-table=openitcockpit.statusengine_tasks \
  --ignore-table=openitcockpit.statusengine_users \
  >/root/openitcockpit_dump_v3_upgrade.sql


Step 3 - Run openITCOCKPIT 4 compatibility check

Notice for users of the i-doit and discovery module

The modules openitcockpit-module-discovery and openitcockpit-module-idoit are not available for openITCOCKPIT 4 yet. If you are using this modules please consider to not upgrade to openITCOCKPIT 4 now.


To ensure that your system is able to get upgraded to openITCOCKPIT 4.x we provide a compatibility check script.

Run the script on your openITCOCKPIT instance.

curl -s https://raw.githubusercontent.com/it-novum/openITCOCKPIT/development/bin/scripts/oitc3to4.sh | sudo /bin/bash

No changes had been made to your system yet!


Do not continue as long as you get any errors or warnings!

Resolve all reported errors and warning. Rerun the compatibility check to ensure that all issues were resolved.



Step 3.1 - Distributed monitoring

It is also required to upgrade all Satellite Systems to openITCOCKPIT 4.

Within the current release it is not possible to connect openITCOCKPIT 3.x satellite systems with an openITCOCKPIT 4.x master system. If you for some reason cannot upgrade your Satellite systems - please do not upgrade your Master system now!

Step 3.2 - Upgrade openITCOCKPIT

Stable SSH connection required!

The upgrade process could take a while. To avoid SSH connectivity issues it is recommended to use tmux or screen.


The compatibility check will print the required commands to upgrade your installation of openITCOCKPIT depending on the installed modules and distribution. The upgrade commands get generated for each system individually.


DO NOT use any other commands to upgrade your system!



Ubuntu Users

Please make sure your system has enabled "universe" repositories

add-apt-repository universe

Debian Users

Please make sure your system has enabled "non-free" repositories

/etc/apt/sources.list
deb http://ftp.de.debian.org/debian/ buster main contrib non-free
deb http://ftp.de.debian.org/debian/ buster-updates main contrib non-free
deb http://security.debian.org/ buster/updates main contrib non-free


Run the commands provided by the compatibility check script to upgrade your openITCOCKPIT.


During the upgrade you may get ask to overwrite some configuration files with newer versions. These are configuration files from the operating system itself so you can choose if you want to keep your old configuration or use the new one. This has no effect on the openITCOCKPIT upgrade.

Ubuntu Xenial and Debian Stretch users only

The upgrade will also upgrade your distribution to the next release. (Ubuntu Xenial to Bionic and Debian Strecht to Buster). If you have installed any custom software or loaded any third party repositories please make sure they are compatible with Ubuntu Bionic/Debian Buster.


Step 4 - Reset user passwords


After the upgrade process, you need to reset all passwords of local users. If you are using LDAP you don't need to do this.

Reset the password of one user

To reset the password of one singe user you could execute the following command:

oitc reset_password --print --enable


The System will generate a random password and send the new password to the users email address.

By passing the --print option the new password will also be printed to the terminal.

Passing the --no-mail option does not send the email with the new password to the user.




Reset all user passwords

To reset the password of all local users you could execute the following command:

oitc update3_to4 --reset-all-passwords

The System will generate a random password for each user and send the new password to the users email address.


Step 5 - Clear Browser cache

Before you login to the new openITCOCKPIT interface you should clear your Browser cache.

Mozilla Firefox: https://support.mozilla.org/en-US/kb/how-clear-firefox-cache

Google Chrome: https://support.google.com/accounts/answer/32050?co=GENIE.Platform%3DDesktop&hl=en

Step 5.1 - Login to the openITCOCKPIT interface

It is recommended to "Refresh" the monitoring configuration. This step ensures that all monitoring related configuration files gets rewritten.


Step 6 - Restart NSTA (Distributed Monitoring only)

To start processing of Satellite check results it is required that you start the NSTA daemon.

systemctl restart nsta

Step 7 - Reboot your system

Especially when upgrading from Ubuntu Xenial or Debian Strech a reboot of the system is recommended.


Update openITCOCKPIT Satellite Systems (Distributed Monitoring only)

Install all system updates first

apt-get update
apt-get dist-upgrade

Execute the openITCOCKPIT Satellite checklist script and follow the instructions

curl -s https://raw.githubusercontent.com/it-novum/openITCOCKPIT/development/bin/scripts/oitcSat3to4.sh | sudo /bin/bash

It is recommended (but not required) to reboot the Satellite System after the upgrade is finished.

Known Issues

All distributions

The modules openitcockpit-module-discovery and openitcockpit-module-idoit are not available for openITCOCKPIT 4 yet. If you are using this modules please consider to not upgrade to openITCOCKPIT 4 now.

Cant login after password reset

If the Password was reset properly and the user still cant login with the message "Invalid username or password", the user must be manually set active in the Database. This can be done with the following SQL Command:

UPDATE users SET is_active = 1 WHERE users.email= 'user@example.com';


Warnings and dependency problems during the upgrade process

During the upgrade you will properly see messages like

Unpacking openitcockpit-module-distribute (4.0.4-20200721154853bionic) over (3.7.3-1ubuntu16.04) ...
dpkg: warning: unable to delete old directory '/etc/openitcockpit/app/Plugin/DistributeModule/Config/Schema': Directory not empty
dpkg: warning: unable to delete old directory '/etc/openitcockpit/app/Plugin/DistributeModule/Config': Directory not empty
dpkg: warning: unable to delete old directory '/etc/openitcockpit/app/Plugin/DistributeModule': Directory not empty
dpkg: warning: unable to delete old directory '/usr/share/openitcockpit/app/Plugin/DistributeModule': Directory not empty

or

Removing php7.0-cli (7.0.33-0ubuntu0.16.04.15) ...
dpkg: php7.0-mysql: dependency problems, but removing anyway as you requested:
 phpmyadmin depends on php-mysql | php-mysqli | php-mysqlnd; however:
  Package php-mysql is not configured yet.
  Package php-mysqli is not installed.
  Package php7.2-mysql which provides php-mysqli is not configured yet.
  Package php7.0-mysql which provides php-mysqli is to be removed.
  Package php-mysqlnd is not installed.
  Package php7.2-mysql which provides php-mysqlnd is not configured yet.
  Package php7.0-mysql which provides php-mysqlnd is to be removed.
 phpmyadmin depends on php-mysql | php-mysqli | php-mysqlnd; however:
  Package php-mysql is not configured yet.
  Package php-mysqli is not installed.
  Package php7.2-mysql which provides php-mysqli is not configured yet.
  Package php7.0-mysql which provides php-mysqli is to be removed.
  Package php-mysqlnd is not installed.
  Package php7.2-mysql which provides php-mysqlnd is not configured yet.
  Package php7.0-mysql which provides php-mysqlnd is to be removed.

You can ignore these messages.

No Satellite check results on the Master Server

In the case that the Master server does not receive any check results from the Satellite systems a "Refresh of monitoring configuration" and a restart of the NSTA maybe resolve the issue.

systemctl restart nsta

Ubuntu Focal (20.04)

openitcockpit-module-snmp-trap

Unfortunately the openitcockpit-module-snmp-trap module can not be installed on Ubuntu Focal at the moment due to Ubuntu related dependency issues.

This bug got resolved with the latest version of Ubuntu Focal.


https://bugs.launchpad.net/ubuntu/+source/net-snmp/+bug/1886658?comments=all


Troubleshooting

Network error

In case you had any connectivity issues you can try to repair the system with:

apt-get install -f

You version of openITCOCKPIT is < 3.7.3!

The update process requires at least openITCOCKPIT 3.7.3. Please consider to check our blogposts (https://openitcockpit.io/blog/#blog) if you are upgrading from an older Version 3, especially when upgrading from Version 3.5 or older. Manual action is required if you want to have your performance graphs available in the newer versions.

Table nagios servicechecks is 36626 MB but /var/lib/mysql only has 16316 MB free disk space.

During the upgrade process all mysql tables get converted from UTF8 to UTF8mb4. This process requires a fair amount of disk space so please don't ignore these errors.

Upgrade from Ubuntu Xenial 16.04 to Bionic 18.04 and openITCOCKPIT v4 failed due to lxd package.

if the upgrade from Ubuntu Xenial fails due to the lxd package and this error:

Old bridge configuration detected in /etc/default/lxd-bridge, upgrading
Unsetting deprecated profile options
Error: unknown command "profile" for "lxc"

Did you mean this?
	Profil

Run 'lxc --help' for usage.
dpkg: error processing package lxd (--configure):
 installed lxd package post-installation script subprocess returned error exit status 1

....

Fehler traten auf beim Bearbeiten von:
 lxd
E: Sub-process /usr/bin/dpkg returned an error code (1)

The frontend of openitcockpit is not visible in the browser

you need to execute the following commands:

apt-get install -f
apt-get remove lxd
apt-get autoremove
reboot

#after reboot you need to trigger the UPGRADE script from itcockpit which was not triggered during the upgrade
/opt/openitc/frontend/bin/scripts/UPGRADE.sh

after that, the Frontend should be visible and the distribution upgrade has finished.

I still use Ubuntu Trusty or Debian Jessie

Please contact sales for commercial support.





  • No labels