Secure Supply Chains with Univention Nubus: Overview of SBOM, VEX and Signatures

Attacks on open source projects have become the norm and therefore also threaten the supply chain of Univention products. To ensure the necessary security and transparency, we rely on signatures, SBOMs and VEX in Univention Nubus. The benefits behind these approaches are explained below.

The “supply chain” of a software includes all parties involved – from writing the source code to commissioning in the data center. In most environments, it consists of a mix of in-house development and externally sourced software.

External software can either be freely available from the internet or provided by commercial vendors. This distinction is relevant for questions of responsibility and licensing terms – but less decisive for the fundamental security of the supply chain.

Almost all risks in the software supply chain can be assigned to two areas:

1. Vulnerability management
   Both before commissioning and during operation, operators need an up-to-date overview of known or newly discovered vulnerabilities. Based on this, measures such as updates or configuration adjustments can be derived. Since new vulnerabilities are continuously published, this process should be automated and carried out regularly.
2. Injected malicious code
   An increasingly common attack vector is the injection of malicious code – either via compromised download servers or directly into the source code. Attackers gain access to code repositories, download servers or infrastructure and manipulate the downloadable software. Operators must therefore ensure that unauthorized modifications are detected before software is used in production.

To keep up with the high dynamics of vulnerability management, automated processes are essential. The following process chain has become established:

1. Identify update options
   Automated mechanisms detect available updates for software products and their dependencies. Operators define rules for which updates are applied automatically and when manual intervention is required.
2. Verify sources
   It is defined which sources are considered trustworthy. Verification is performed both at the level of approved source servers and the publishers of individual software packages. Software from untrusted sources is not used.
3. Identify vulnerabilities
   Vulnerabilities are reported after a software release and can therefore affect both already deployed and newly introduced software. Operators therefore carry out regular analyses of all software in operation in order to identify risks and initiate measures. These analyses are also applied to newly introduced software.
4. Select measures
   In the simplest case, vulnerabilities can already be addressed by installing updates (step 1). However, since this is not always possible, operators often use dashboards in risk management to identify the need for action. Alternatives to updates can include configuration changes, temporary deactivation of functions or the targeted detection of attacks exploiting these vulnerabilities.

In the following, the most important technical measures required for this process are described.

Signatures have become the standard mechanism for verifying trustworthiness – regardless of whether software is installed in UCS as a “.deb” package, as a container or on Windows. While implementations differ in detail between various package formats and repository servers, they are always based on a key pair: a private key held by the software creator and a public key that operators can use for verification, for example based on the GPG implementation.

When providing software, the software creator – for example Univention – uses the private key to sign either the software packages themselves or, as in Debian repositories, the checksums. These signatures are part of the software distribution. When operating Nubus with Univention Corporate Server, the mechanisms implemented by “apt” are used, while for Nubus for Kubernetes, containers and Helm charts are signed (see below). Since both methods are widely used, automated verification with standard tools is possible.

A complete verification includes not only signatures but also checksums. These are generated by the package creator using standardized hash algorithms. The operator applies the same algorithm and normally obtains the same result, i.e. the same checksum. If an attacker manages to modify a package on the download server, the checksum will differ and the attack will be detected. Thanks to the signature of the checksums, manipulations of these would also be detected.

Keeping track of which software and especially which libraries are currently deployed can be more or less complex depending on the environment. However, such a “software inventory” is essential for assessing potential vulnerabilities.

When Nubus is deployed on Univention Corporate Server (UCS), the main components can still be determined relatively easily thanks to the Debian-based package management using tools such as “dpkg” and “apt”. For container images used via Docker on UCS or in Nubus for Kubernetes, this is much less obvious. Since there are different methods of packaging software into containers, different analysis tools are required to capture all contents.

Nubus for Kubernetes therefore provides the contents of container images in a standardized format as a so-called “Software Bill of Materials” (SBOM). This makes it possible to easily and automatically determine which software in which version is contained in each container image – including the libraries used.

In addition to the information required for vulnerability management, both Debian packages of UCS and SBOMs of container images also contain license information that can be used for compliance management.

Thanks to SBOMs, operators gain a detailed insight into the software and libraries obtained from trusted sources. On this basis, automated comparison with vulnerability databases is possible, which typically reference “Common Vulnerabilities and Exposures” (CVE) IDs.

Automated vulnerability scanners usually compare the versions of the software used with versions affected by vulnerabilities. If the deployed version is affected, they report a potential vulnerability.

Such findings are often valid and must always be reviewed. However, they are not infrequently so-called “false positives”. The most common causes are:

1. No version change with backported fixes
   To maintain compatibility, especially in long-term support environments, security fixes are selectively backported to older versions. Since the version number does not change or only slightly, scanners report a vulnerability even though it has already been fixed. This is particularly common with Debian packages.
2. No exploitability of the vulnerability
   In many configurations, not all functions of a software or library are used. If a vulnerability only occurs in a function that is not used in the specific environment, it is effectively not exploitable. Since scanners lack this contextual knowledge, they still report a risk. If an update is associated with complications or incompatibilities, it may be more reasonable to accept a non-exploitable vulnerability.

As a software manufacturer, Univention systematically evaluates such cases and decides whether a vulnerability is exploitable and whether a fix is provided as an update or backport – this process is referred to as “triaging”. To make this information available to operators, the “Vulnerability Exploitability eXchange” (VEX) format was defined. It complements SBOMs with information on vulnerabilities that could not be resolved by a simple version update. Univention provides VEX information for all container images of Nubus for Kubernetes.

As with software packages and container images, it is also important to verify signatures for VEX and SBOMs. Otherwise, attackers could manipulate this information to hide vulnerabilities. Signature verification reliably detects such changes.

With Nubus for Kubernetes, Univention covers all aspects of supply chain security. All components (“artifacts”) of a release – including container images, Helm charts, SBOMs and VEX – are signed. The implementation is based on the widely used “Sigstore” project, which is already integrated into many Kubernetes environments. Details of the implementation and examples of how to verify the authenticity of the artifacts can be found in the “Supply Chain Security” chapter of the Nubus for Kubernetes Operation Manual.

The German Federal Office for Information Security (BSI) addresses supply chain security both in the IT-Grundschutz catalog and in various technical guidelines. All implementations of Nubus for Kubernetes are designed to implement these requirements in a practical manner.

Good entry points into the extensive information provided by the BSI include:

1. The technical guideline BSI TR-03183 on requirements for manufacturers and products, based on the supply chain security defined in the “Cyber Resilience Act” (CRA). The guideline consists of three parts, of which part 2 covers SBOMs and part 3 covers VEX.
2. The requirements for software manufacturers such as Univention in the BSI IT-Grundschutz catalog, described in CON.8 “Software Development”. This includes, for example, CON.8.A6 (obligation to use trustworthy software components) and CON.8.A8 (provision of checksums and signatures).
3. The requirements for operators of software, described in the “OPS” modules, such as OPS.1.1.3.A10 (verification of signatures before using software packages) and OPS.1.1.3.A15 (handling of updates to protect against newly discovered vulnerabilities).

SBOMs not only contain the version numbers of the software used, but also the respective license information. These may be open source licenses or proprietary licenses. This allows operators to perform compliance checks to determine whether the license terms are acceptable for their organization and can be complied with. Thanks to the standardized SBOM format, these checks can be automated.

A typical approach is to define acceptable licenses within the organization and validate the contents of SBOMs against them. If new software is added or an existing software changes its license, an automated check detects if the new license does not comply with the defined requirements. This allows the organization to act before the software is deployed and the license terms potentially take effect.

Freely available open source projects unfortunately vary greatly in their maturity regarding supply chain security. The provision of signatures and checksums has become common, as these are automatically generated by most build processes. However, SBOMs or even VEX information are rare and often not meaningful for individual software libraries.

Open source software manufacturers such as Univention therefore take on a dual role in supply chain security: on the one hand, they secure the supply chain of the software used in their products, and on the other hand, they themselves are part of such a supply chain for their customers. As a manufacturer, they close a gap by generating, for example, complete SBOMs from the base information provided by open source projects.

Univention takes this responsibility seriously and ensures that software included in delivered products originates from trustworthy sources, can be maintained with security updates and is provided under OSI-compliant open source licenses. The techniques and automations described in this article are also used internally.

As a Univention customer, you receive the assurance as part of the subscription agreement that these security measures have already been implemented.