FossID Documentation

Workbench Server Update Instructions

Assumptions

  • This is an update from version 22.2.x or above. To update from an earlier version, please update to 22.2.x first.

  • Target version is 2025.1.0

  • Existing Workbench is installed at /fossid
  • Database tables permissions and role_permission have not been modified.

Package changes

As of version 23.3 we have added the ability to rotate the FossID Workbench logs

OS requirements

FossID Workbench is supported on:

  • Debian 11
  • Debian 12
  • Ubuntu 20.04
  • Ubuntu 22.04
  • Ubuntu 24.04
  • RedHat Enterprise Linux 8 and 9
  • RedHat Enterprise Linux Server 8 and 9

The minimum PHP required version is 8.2.

The minimum required version of the database server is MySQL Server 8.0 or MariaDB 10.6. However, we recommend that all sub-systems are at least in their oldest still maintained version.

Optional preparation for updating to 22.2

Updating a Workbench installation with a large database to 22.2 or to current version coming from version older than 23.1 might take a very long time due to a specific database change. During the update it is not possible to use the webapp.

It is possible to prepare for the update by connecting to the database and executing the following statement:

ALTER TABLE "file_client_results" MODIFY "file_license" mediumtext NULL;

Once this is completed, which is the part that will take a long time, the actual update will not take longer than usual.

PHP prerequisites

If your FossID server is running on Debian 11 or Ubuntu 20.04 or Ubuntu 22.04, Redhat 8 or RHEL Server 8 you need to add a repository with a php version equal or greater than 8.2 first:

Debian 11

sudo apt update
sudo apt install lsb-release apt-transport-https ca-certificates -y
sudo wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/php8.2.list
sudo apt update
sudo apt remove php7\*
sudo apt remove php8.0\*

Ubuntu 20.04 or Ubuntu 22.04

sudo add-apt-repository ppa:ondrej/php -y
sudo apt update
sudo apt remove php7\*
sudo apt remove php8.0\*

For both Debian and Ubuntu, modify the apt remove command to reflect the version of php that needs to be removed.

NOTE: On some Debian or Ubuntu system due to the way the package manager works some php packages are not installed for the version of php in use. Please check the troubleshooting page.

RedHat

Epel repository on RedHat

RedHat 8

sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm -y

RedHat 9

sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm -y
Remis’ RPM Repository

RedHat 8

sudo yum install -y https://rpms.remirepo.net/enterprise/remi-release-8.rpm

RedHat 9

sudo yum install -y https://rpms.remirepo.net/enterprise/remi-release-9.rpm
Install packages

Set the desired version of PHP (minimum required version is 8.2):

sudo yum module reset php
sudo yum module enable php:remi-8.2 -y
RedHat 8
sudo yum install php-cli php-fpm php-process mariadb-server mariadb php-mysqlnd php-zip php-xml php-curl php-ldap php-mbstring php-intl vim libxslt unzip java-11-openjdk-headless git p7zip p7zip-plugins perl -y
RedHat 9
sudo yum install php-cli php-fpm php-process mariadb-server mariadb php-mysqlnd php-zip php-xml php-curl php-ldap php-mbstring php-intl vim libxslt unzip java-11-openjdk-headless git p7zip p7zip-plugins perl -y

MySQL/MariaDB upgrade

Due to the older version in different distributions of MySQL or MariaDB, we recommend upgrading to at least MySQL 8.0 using the official guides at https://dev.mysql.com/doc/refman/8.0/en/installing.html or, for MariaDB 10.6 from https://mariadb.com/kb/en/binary-packages/.

NOTE: It is recommended to explicitly set these values for character set and collation in your MySQL/MariaDB config file:

character-set-server     = utf8mb4
collation-server         = utf8mb4_general_ci

For MySQL Replication the parameter default_collation_for_utf8mb4 must be set to utf8mb4_general_ci. More details here: https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_default_collation_for_utf8mb4

A value of at least 64M needs to be set for max_allowed_packet under [mysqld] tag in the corresponding file for your mysql server distribution (e.g. /etc/mysql/my.cnf or /etc/my.cnf). See below reference:

[mysqld]
max_allowed_packet = 64M

This may vary for each Linux distribution and mysql server distribution. Please review the documentation for your corresponding Linux and mysql version distribution.

Restart mysql:

sudo service mysql restart

Access Deliverables

Access info to the FossID deliverables is provided in the delivery mail.

Install the package

DEB installer

sudo apt install ./fossid-release_regular_amd64-{VERSION}.deb -y
sudo systemctl restart php$(php -r 'echo PHP_MAJOR_VERSION.".".PHP_MINOR_VERSION;')-fpm.service

RPM installer

sudo yum localinstall ./fossid-release_regular.x86_64-{VERSION}.rpm -y

Update fossid.conf

As of FossID Workbench 23.3, changes need to be done in fossid.conf if you were previously using Microsoft OAuth2 login as some parameters were renamed/added to support multiple identity providers:

  1. “webapp_microsoft_oauth2_login” has been renamed to “webapp_oauth2_login”
  2. “webapp_microsoft_oauth2_fallback_local_login” has been renamed to “webapp_oauth2_fallback_local_login”.
  3. parameter webapp_oauth2_provider=”Microsoft” needs to be added to fossid.conf.

As of FossID Workbench 21.2 it is no longer necessary or recommended to migrate your fossid.conf settings. The Workbench will work with your existing pre-21.2 configuration.

If you still want to migrate your old settings to the new template for any reason this is how:

  1. Backup the existing /fossid/etc/fossid.conf file to e.g. fossid.conf_bak
  2. Rename /fossid/etc/fossid.conf.dist to /fossid/etc/fossid.conf
  3. Modify the new /fossid/etc/fossid.conf file to match your configuration in the fossid.conf_bak

Configure Workbench URL

This information is used to generate correct absolute URLs in emails:

webapp_base_url = https://mycompany.com/index.php

Save your fossid.conf file.

NOTE: The changes to configuration is immediate, no restart is required.

Post install actions

Update database

Starting from 22.2 version new unique constraints are added at database level to avoid duplicate data.

In order to help identifying and fixing eventual existing duplicates the following command needs to be executed from /fossid/webapp directory:

cd /fossid/webapp
sudo -u www-data php bin/console.php enforce_data_integrity

An interactive command-line interface will guide you trough the process. For duplicated licenses or components the script will help you fix the duplicates by reassigning the identifications to one record and removing the duplicated one (so no identification is lost). For projects/scans/users duplicated will be fixed by adding some suffix.

The system checks if there is a duplicates in DataBase tables: 
'licenses'; 
'projects'; 
'components'; 
'scans'; 
'users'; 
----------------------- 
The system didn't find duplicates. Do you want to add UNIQUE CONSTRAINTS to the tables? (defaults to yes)
  [0] yes
  [1] no
  > 0

In the best scenario there are no duplicates found and unique constraints can be directly added. If indeed duplicates are found the following message is displayed:

"The system found duplicates in the next tables:"
'projects'
'licenses'
----------------------- 
Do you want to clean duplicates? (defaults to yes)
  [0] yes
  [1] no
 > 0

The following tables and columns are used for checking uniqueness of the data:

  Table       unique by    Column(s)
 'projects'                'project_code' 
 'licenses'                'identifier' 
 'components'              'name' and  'version' 
 'scans'                   'code', 
 'users'                   'username'

Please follow instructions displayed by interactive CLI to fix any duplicates found. Last step is adding unique constraints to the those tables. If the unique keys already exists an error message is displayed:

The system didn't find duplicates. Do you want to add UNIQUE CONSTRAINTS to the tables? (defaults to yes)
  [0] yes
  [1] no
 > 0
The system tries to add UNIQUE CONSTRAINT 'uc_identifier' to table: 'licenses' on column: `identifier`.
SQLSTATE[42000]: Syntax error or access violation: 1061 Duplicate key name 'uc_identifier'

No other action needs to be taken as the unique key is already in place.

A one time action is required for migrating component identifications made before release 20.1:

Before release 20.1 it was possible to assign as identification just one component per file identified. Run one time the below command to migrate any old component identification from column “component_id” in table “identifications” to table “identification_component”. The column “identifications.component_id” is removed in 24.3 release by dbupdate.php script.

sudo -u www-data php bin/console.php migrate_legacy_identifications --auto_migrate

A one time action is required when updating from a version before 23.1.x:

sudo -u www-data php bin/console.php split_file_client_results

The table file_client_results will be split in one table per scan. This will alleviate database maintenance by immediately regaining disk space from MySQL when a scan is deleted by dropping the file_client_results_x table associated with deleted scan. Please inspect if tables with name like file_client_results_SCAN_ID were created.

The initial table file_client_results still exists with name old_file_client_results and can be dropped manually after checking if new tables were created and data migrated.

After fixing duplicates and running split_file_client_results command continue the regular upgrade process.

Navigate to the setup folder:

cd /fossid/setup/database

Verify with the below query if row_format is set as DYNAMIC or COMPRESSED in all tables, otherwise you will need to manually change them with the ALTER TABLE statement more below.

To verify if row_format is set to DYNAMIC or COMPRESSED on each table:

SELECT table_name, row_format FROM information_schema.tables WHERE table_schema=DATABASE();

To manually modify the table if needed (in the example below is set to DYNAMIC):

ALTER TABLE [table_name] ROW_FORMAT=DYNAMIC;

Set up indexes and verify database structure:

sudo -u www-data php dbupdate.php /fossid/etc/fossid.conf

Verify the existence of all the required data (such as roles and permissions) and remove old roles/permissions:

sudo -u www-data php db_info_update.php /fossid/etc/fossid.conf

Update the license database:

sudo -u www-data php licenseupdate.php --no-update --no-disable /fossid/etc/fossid.conf

Note the arguments --no-update and --no-disable are optional:

  • --no-update: use this argument if you made any edits to the license texts of licenses that are part of the default database and wish to preserve these edits. This will not stop wrong license identifiers being fixed, see –no-fix-idents.

  • --no-disable: use this argument if you manually added licenses that are not part of the default database and wish to be able to continue to identify files as these licenses, or wish to continue to identify files with identifiers that have been marked as deprecated according to the SPDX standard.

  • --no-fix-idents: the –no-fix-idents option will prevent this utility from updating bad or old identifiers found in the database. It is not recommended to enable this option unless –no-insert is also used as some licenses that already exists may be inserted again.

  • --dry-run : do not make any modifications, only show what will be done

For complete licenseupdate.php manual please run:

php licenseupdate.php --help

Optional PURL and CPE update from KB

PURL field was added in Workbench 23.1. In order to try filling this field from KB for already existing components in Workbench database run this command. Also this command will fill CPE info from KB if local Component has this field empty:

cd /fossid/webapp
sudo -u www-data php bin/console.php add_purl_and_cpe

Update OSS Review Toolkit for Dependency Analysis

There are two tools: FossID-DA or OSS Review Toolkit that can be used to provide information on package dependencies and their license information right in the FossID Workbench user interface. Using the Dependency Analysis feature, you can get a better insight into the licenses your software needs to be compliant with. FossID also provides API for the Dependency Analysis so it can be included in your Continuous Integration pipeline.

Since Workbench 24.3 release the FossID-DA tool is the default dependency analysis tool. For more details about FossID-DA please see the dedicated documentation section here

ORT has recently gone through an API change and this version of FossID Workbench requires that you update to the latest version. If you installed the previous version of ORT using the installation package from your delivery portal, you will need to uninstall it and delete the /fossid/lib/ort folder first. Then download the new ORT version (22.2) from the delivery portal and install it.

To uninstall from RHEL, run:

sudo yum remove fossid-ort
sudo rm -rf /fossid/lib/ort

On Debian/Ubuntu:

sudo apt remove fossid-ort
sudo rm -rf /fossid/lib/ort

If you built ORT from source code, pull the latest changes and build it again.

See the Installation - Dependency Analysis for additional details.

Changes to the nginx configuration

In the 23.3.0 release new nginx locations and rewrites have been added. Please consult your configuration and synchronize it with the example in the /fossid/setup/templates/nginx.conf file.


Changes required when upgrading PHP

If you have upgraded PHP, please make the following changes.

Open the /etc/nginx/nginx.conf and look for the following section:

location = /index.php {
    # If any other php version than 8.2 is used, please update this path
    fastcgi_pass unix:/run/php/php8.2-fpm.sock;
    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param SCRIPT_NAME $fastcgi_script_name;
}

Change the fastcgi_pass unix:/run/php/php8.2-fpm.sock; to point to the right version of php. For example, if php version is 8.3, the line should look like this:

fastcgi_pass unix:/run/php/php8.3-fpm.sock;

Restart nginx service:

sudo service nginx restart

Edit /etc/php-fpm.d/www.conf and make sure the following configuration is set:

user = www-data
group = www-data
listen = /run/php/php8.2-fpm.sock
listen.owner = www-data
listen.group = www-data
listen.mode = 0660
;listen.acl_users = apache,nginx   <-- make sure it's commented out.

Change the listen = /run/php/php8.2-fpm.sock to point to the right version of php. For example, if php version is 8.3, the line should look like this:

listen = /run/php/php8.3-fpm.sock

Change the group ownership of /var/lib/php and then restart the php-fpm:

sudo chgrp www-data /var/lib/php/*
sudo systemctl restart php-fpm

Log rotation

As of Fossid Workbench 23.3 we have added the files needed for rotating the logs in /fossid/logs. The configuration can be tuned to the usage needs by editing the file /etc/logrotate.d/fossid-workbench.

Remove old crontab entries

As of FossID 21.2, crontab is no longer used for background tasks (Vulnerability monitoring and scan queue handling). If you are upgrading an installation that had background tasks configured, run:

sudo crontab -e

and delete any entries similar to:

* * * * *  /fossid/www/webapp/scans_queue_consumer.sh
0 1 * * *  /fossid/www/webapp/cpe_update_check.sh
0 0 * * *  /fossid/www/webapp/monitor.php --check v --n

Optional: Whitelisting

As of FossID 20.2, whitelisting is not enabled by default. To enable it, add the following parameters to webapp_cli_command in fossid.conf: --fields +mid e.g.:

webapp_cli_command=/fossid/bin/fossid-cli --config /fossid/etc/fossid.conf --fields +mid

Note that files that were already scanned before whitelisting was enabled will have to be re-scanned in order for adding whitelisting rules from those files to work.

FossID Workbench Help, version 2025.1.0