Leer en español · Lire en français · Auf Deutsch lesen
Section 8.6 of 8.8 · Intermediate · 23 min read
Software supply-chain security
Video lesson
5 min · narrated, with captionsListen only (audio)
Read the transcript
Most of the code in a modern service wasn't written by the team that runs it. This lesson shows why everything you build on is part of your attack surface, and how architects reduce that risk.
A restaurant doesn't grow its own flour. It trusts suppliers. If a supplier's flour is contaminated, every restaurant that bought it is affected, however clean its own kitchen is. So good restaurants keep a list of every ingredient and where it came from. They buy from suppliers who seal and label their deliveries, and they check the seals on arrival. Software is built the same way, from hundreds of components written by other people and assembled by build machines.
Here is the chain: a developer, the source repository, the build system, a registry that stores the result, and deployment. Every arrow is a trust relationship, and each famous incident abused a different one. Open-source dependencies feed the build. In 2018, a popular package called event-stream was handed to a volunteer, who added a malicious dependency. In 2021, Log4Shell was an honest but serious flaw in a logging library buried deep in countless systems. In 2024, the xz utils attack came through the source and release process. After about two years of patient, useful contributions, a co-maintainer shipped release files containing a hidden backdoor. It was caught early. In 2020, attackers inside SolarWinds tampered with its build system, so the company's genuinely signed updates carried a backdoor. A signature proves who built something, not that the build was clean. And registries are abused with look-alike package names, called typosquatting, and with public packages named like a company's internal ones.
A software bill of materials, or SBOM, is a machine-readable list of the components in your software, with versions and exact identifiers. It doesn't make software safer by itself. It makes questions answerable. Generate it during the build, from what was actually resolved and packaged, not from the list a developer wrote. Store it with the artefact, and feed it to a service that checks every stored list again whenever a new vulnerability is published. When Log4Shell was announced, organisations with an accurate inventory answered the question in hours. Others spent weeks.
This is a hardened pipeline from commit to production. It starts with reviewed source on protected branches, and dependencies locked to exact versions, with a hash of each. The build is isolated, created for one job and destroyed afterwards, with all its inputs declared. It produces the bill of materials. The result is signed without any long-lived key. A certificate that lasts minutes names the build's identity, and the signature is recorded in a public log that can only be added to. The image is stored by its digest. Before production runs it, an admission check verifies the signature and where it came from. Only what the expected pipeline built from reviewed source is allowed to run.
Commit lockfiles, which record the exact versions with a hash of each, and install from them in every build. Pin build tools too. In 2025, a popular build action's version tags were moved to malicious code, affecting thousands of repositories that referred to it by tag. Pinning without updating creates the opposite risk, so let a bot propose small updates often. Waiting a few days before adopting a routine release helps, because most malicious releases are pulled within days. Security fixes go fast. And scanners produce hundreds of findings. Prioritise by whether a flaw is being exploited, whether your code can reach it, and whether the service faces the internet.
Regulation is pushing in the same direction. The EU Cyber Resilience Act sets cybersecurity requirements for hardware and software products with digital elements placed on the EU market. Its requirements are architectural: secure default configuration, a limited attack surface, security updates, and an SBOM covering at least top-level dependencies. It applies in stages. Its obligations to report actively exploited vulnerabilities and severe incidents apply from the eleventh of September 2026, and most other obligations from the eleventh of December 2027. This is general information, not legal advice. Whether a particular product is in scope is a legal question for each organisation, to confirm with the official texts and qualified advisers.
The full section tells the four incidents in detail, and has the build integrity levels and a checklist for service teams. Next, in Section eight point seven, we look at privacy by design. See you there.
What you need to know
- Most code in a modern service comes from dependencies, build tools, CI actions and base images; every one of them, and everyone who can change them, is part of the attack surface.
- event-stream (2018) abused maintainer trust, SolarWinds (2020) a compromised build system, Log4Shell (2021) a vulnerable library buried deep in dependency trees, and xz utils (2024) a patient takeover of a release process.
- An SBOM in SPDX or CycloneDX format lists components with exact identifiers so that exposure to a new vulnerability can be found by query; US Executive Order 14028 (2021) and the EU Cyber Resilience Act made SBOMs mainstream.
- SLSA (version 1.2) defines Build levels L0 to L3, ending with hardened, isolated builds, and Source levels up to two-party review; provenance records who built an artefact from what.
- Sigstore's keyless signing ties a short-lived certificate to a CI identity and records signatures in a public transparency log; admission checks verify signatures and provenance before deployment.
- Lockfiles with hashes, pinning CI actions and images by digest, small automated updates with a short delay for new releases, prioritised scanning and ephemeral, hermetic builds reduce the risk in daily work.
- The CRA's reporting obligations apply from 11 September 2026 (24-hour early warning, 72-hour notification) and most other obligations from 11 December 2027.
Why dependencies are an attack surface
Most of the code in a modern service was not written by the team that runs it. A small web service may declare twenty direct dependencies and, through their dependencies in turn, install several hundred packages; the container it ships in adds an operating-system userland, and the pipeline that builds it runs compilers, plug-ins and CI actions written by still more people. Section 0.3 introduced dependencies as a saving in effort with a maintenance cost. This section looks at their security cost: every one of those components, and every person and system able to change them, is part of your attack surface.
A software supply-chain attack compromises something you consume in order to reach you, usually many victims at once. It can arrive in four broad ways. Malicious code from a trusted source: a maintainer's account is phished, a maintainer turns hostile, or a project is handed to a stranger. Malicious look-alikes: typosquatting publishes reqeusts next to requests; dependency confusion, demonstrated by Alex Birsan in February 2021, publishes a package on the public registry with the same name as a company's internal package and a higher version number, which some build tools then prefer. Tampering in the pipeline: the source is fine but the build system, the signing keys or the registry are compromised. Honest vulnerabilities: no attacker was involved in writing the code, but a serious bug in a widely used transitive dependency exposes everyone who ships it.
Recent years have shown how fast the first kind spreads. In March 2025 the version tags of a popular GitHub Action, tj-actions/changed-files, were moved to a malicious commit that printed CI secrets into build logs, affecting thousands of repositories that referred to the action by tag. In September 2025 a maintainer of widely used npm packages such as chalk and debug was phished through a fake npm support site, and malicious versions were published and removed within hours; later the same month a self-propagating worm, nicknamed Shai-Hulud, stole npm tokens and used them to publish infected versions of further packages.
The supply chain and the trust relationships attackers abuse
Four incidents, told accurately
event-stream (2018). event-stream was a popular npm package, downloaded about two million times a week, whose original author no longer had time for it. In September 2018 he handed publishing rights to a volunteer who had offered to help. The new maintainer added a dependency, flatmap-stream, whose published package contained an encrypted payload that activated only inside the build of Copay, a bitcoin wallet application, and tried to steal wallets with large balances. A developer noticed the suspicious code in November 2018. Nothing in npm's model had been broken: a trusted name had simply changed hands.
SolarWinds (2020). Attackers, later attributed by the US government to Russia's foreign intelligence service, gained access to the build environment of SolarWinds, the maker of the Orion network management product. An implant (named SUNSPOT by investigators) watched for Orion builds and swapped one source file for a backdoored version during compilation, then put the original back. The resulting updates, published between March and June 2020, were built by SolarWinds' own pipeline and signed with its genuine certificate. About 18,000 customers installed them; the attackers then chose a much smaller number of government agencies and companies for follow-on intrusion. FireEye discovered the campaign in December 2020 while investigating its own breach. The lesson for architects: a signature proves who built something, not that the build was clean.
Log4Shell (2021). CVE-2021-44228, disclosed on 9 December 2021 and scored 10.0 under CVSS, was a flaw in Log4j 2, the dominant Java logging library. Log4j resolved lookups written inside log messages, including JNDI lookups that could load a class from a remote server, so an attacker who could get a string such as a username or a request header logged could run code on the server. No attacker had touched the supply chain; the damage came from how widely, and how deeply, the library was embedded. Organisations spent weeks finding where they used it, several follow-up releases were needed to close related flaws, and those with an accurate inventory answered the question in hours.
xz utils (2024). xz is a compression library used, through other libraries, by the OpenSSH server on several Linux distributions. Over about two years an account called Jia Tan contributed useful work and, helped by other accounts pressing the overworked original maintainer to accept help, became a co-maintainer. Versions 5.6.0 and 5.6.1, released in February and March 2024, contained a backdoor that would let the holder of a specific private key run commands through SSH. The malicious code was hidden in binary test files and activated by a modified build script that existed only in the release tarballs, not in the public repository. On 29 March 2024 Andres Freund, a developer at Microsoft, disclosed it after investigating why SSH logins on a test system used unusual CPU and took about half a second longer; it was tracked as CVE-2024-3094. Because it was caught early, it had reached mainly development and rolling-release distributions rather than stable ones.
What each incident teaches
| Incident | What happened | Link attacked | Controls that would have helped |
|---|---|---|---|
| event-stream (npm), 2018 | A new volunteer maintainer, handed the package by its tired original author, added a dependency (flatmap-stream) carrying an encrypted payload aimed at one bitcoin wallet application. | Maintainer trust and transitive dependencies | Lockfiles and a delay before adopting new versions; review of new transitive dependencies; funding and succession for critical packages. |
| SolarWinds Orion, 2020 | Attackers inside the vendor's network implanted code that swapped a source file during the build, so legitimately signed updates contained the SUNBURST backdoor. | Build system | Isolated, ephemeral, monitored builds; provenance; reproducible builds to compare outputs; least privilege for the network management tool itself. |
| Log4Shell (Log4j 2), 2021 | A long-standing feature that resolved JNDI lookups in logged text let any attacker who could get a string logged run code on the server. | A vulnerable, deeply nested dependency | An SBOM to find every affected system in minutes; egress filtering; fast, rehearsed patching. |
| xz utils, 2024 | After about two years of patient contributions, a co-maintainer shipped release tarballs with a hidden backdoor that targeted OpenSSH on some Linux distributions. | Source and release process | Building from the reviewed repository, not from hand-made tarballs; reproducible builds; scrutiny of binary test files and build scripts. |
Software bills of materials
A software bill of materials (SBOM) is a machine-readable list of the components in a piece of software, with versions, identifiers and the relationships between them. It does not make software safer by itself; it makes questions answerable. When the next Log4Shell is announced, a searchable store of SBOMs for every deployed artefact turns are we affected, and where?
into a query.
Two formats dominate. SPDX, from the Linux Foundation, began as a licence-compliance format; version 2.2.1 is the international standard ISO/IEC 5962:2021, and SPDX 3.0 (2024) adds profiles for security, builds and AI. CycloneDX, an OWASP project designed for security use cases, is standardised by Ecma International as ECMA-424; version 1.7 was released in October 2025. Both can identify components with a package URL (purl), such as pkg:npm/express@4.21.2, which is what vulnerability databases match against. The companion idea of VEX (Vulnerability Exploitability eXchange) lets a supplier state that a listed vulnerable component is not exploitable in its product, for example because the vulnerable function is never called, which cuts the noise for customers.
Policy pushed SBOMs into the mainstream. US Executive Order 14028, Improving the Nation's Cybersecurity, signed on 12 May 2021, set out software supply-chain measures for software sold to the federal government, including providing purchasers with an SBOM; the National Telecommunications and Information Administration then published the minimum elements of an SBOM (supplier, component name, version, other unique identifiers, dependency relationships, author of the SBOM data and a timestamp). US policy has since moved away from uniform mandates: in January 2026 the Office of Management and Budget's memorandum M-26-05 rescinded the earlier requirement for suppliers to attest to secure development practices, leaving agencies to set risk-based requirements and to ask for SBOMs where they choose. In the EU, the Cyber Resilience Act described below makes an SBOM part of the technical documentation for products with digital elements.
Generate SBOMs during the build, from what was actually resolved and packaged, rather than from the manifest a developer wrote; store them alongside the artefact, keyed by its digest; and feed them to a service that re-checks every stored SBOM when a new vulnerability is published.
{
"bomFormat": "CycloneDX",
"specVersion": "1.6",
"metadata": {"timestamp": "2026-10-20T09:14:02Z",
"component": {"bom-ref": "orders-service", "type": "application",
"name": "orders-service", "version": "4.12.0"}},
"components": [
{"bom-ref": "log4j-core", "type": "library", "name": "log4j-core",
"group": "org.apache.logging.log4j", "version": "2.24.3",
"purl": "pkg:maven/org.apache.logging.log4j/log4j-core@2.24.3",
"hashes": [{"alg": "SHA-256", "content": "9f2c..."}]},
{"bom-ref": "jackson-databind", "type": "library", "name": "jackson-databind",
"version": "2.18.2",
"purl": "pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.18.2"}
],
"dependencies": [
{"ref": "orders-service", "dependsOn": ["log4j-core", "jackson-databind"]}
]
}SLSA: levels for build integrity
SLSA (Supply-chain Levels for Software Artifacts, pronounced salsa) is a framework, now maintained under the Open Source Security Foundation, that describes incremental levels of protection against tampering. Version 1.0 (2023) defined a Build track; version 1.2 restructured the specification into several tracks and added a Source track. The central artefact is provenance: a signed statement, usually in the in-toto attestation format, of which build platform produced an artefact, from which source revision, with which build definition and parameters. A consumer verifies provenance against expectations such as built by our CI from the main branch of github.com/fooddelivery/orders.
The Source track has four levels: version control (L1), continuous, immutable history with provenance for each revision (L2), continuously enforced technical controls such as protected branches (L3), and two-party review of every change to protected branches (L4). For most organisations the realistic target is Build L3 on a shared, well-run build platform plus Source L3 or L4 for important repositories; the levels say what a platform must guarantee, and a platform team can meet them once for every service.
SLSA Build track levels
| Level | Name | What it guarantees |
|---|---|---|
| Build L0 | No guarantees | No requirements: the absence of SLSA. |
| Build L1 | Provenance exists | The build produces provenance describing how the artefact was built. It catches mistakes but is easy to forge. |
| Build L2 | Hosted build platform | A hosted build service generates and signs the provenance, protecting against tampering after the build. |
| Build L3 | Hardened builds | The build platform isolates builds from each other and keeps signing material out of reach of user-defined build steps, protecting against tampering during the build. |
Signing and provenance with Sigstore
Signing artefacts has long been possible, but managing long-lived signing keys was so burdensome that few open-source projects did it, and stolen keys were themselves a risk. Sigstore, launched in 2021 and now part of the Open Source Security Foundation, makes signing cheap through keyless signing. A CI job proves its identity to Fulcio, Sigstore's certificate authority, with an OpenID Connect token; Fulcio issues a certificate valid for a few minutes that names that identity, such as a specific workflow in a specific repository; the tool signs the artefact and records the signature in Rekor, a public append-only transparency log; the ephemeral private key is then discarded. Verifiers check the signature, the identity in the certificate and the log entry, instead of trusting a key that might have leaked.
The cosign tool applies this to container images and other artefacts, and package ecosystems have adopted the same ideas: npm can publish packages with provenance statements linking them to the source repository and build (2023), and PyPI supports trusted publishing from CI without long-lived tokens (2023) and attestations based on Sigstore (2024). At deployment, an admission controller in Kubernetes, such as Sigstore's policy controller or Kyverno, can refuse any image that is not signed by the expected identity or lacks the expected provenance. Remember SolarWinds: signing and provenance prove where something came from; only build isolation, review and detection make what came out trustworthy.
A hardened pipeline from commit to production
Lockfiles, pinning and automated updates
A manifest such as package.json or pyproject.toml states acceptable version ranges; a lockfile (package-lock.json, poetry.lock, uv.lock, go.sum, Cargo.lock) records the exact versions resolved, usually with a cryptographic hash of each. Committing the lockfile and installing from it in CI (npm ci, pip install --require-hashes) means that every build uses the same reviewed bytes, and that a package republished with different content fails the hash check. Pinning applies to build tooling too: refer to CI actions by full commit hash, not by a tag that its owner, or an attacker, can move, which is exactly what made the tj-actions incident spread; refer to base images by digest.
Pinning without updating creates the opposite risk: old versions accumulate known vulnerabilities and upgrades grow too large to attempt. Update bots such as Dependabot and Renovate open small, frequent pull requests that CI tests automatically. A useful refinement is a minimum release age, waiting a few days before adopting a routine new version, because most malicious releases are detected and pulled within hours or days; security fixes are exempt and fast-tracked.
Software composition analysis (SCA) tools, such as OWASP Dependency-Check, Trivy, Grype or commercial services, match your dependencies against vulnerability databases. Their weakness is noise: hundreds of findings, most in code paths you never call. Prioritise with exploit data (is it in the CISA Known Exploited Vulnerabilities catalogue, what is its EPSS score?), with reachability analysis where your tools offer it, and with exposure (is the service internet-facing?). Section 8.8 covers vulnerability management as a process.
Containers and build integrity
A container image carries an operating-system userland as well as your code, and every package in it is part of the supply chain. Start from minimal base images (distroless, Alpine, or slim distribution images), which contain fewer packages to be vulnerable; rebuild images regularly so that base-image fixes arrive even when your code has not changed; scan images in CI and again continuously in the registry, because new vulnerabilities are published against old images every day; and deploy images by digest so that what was scanned is what runs.
The build system is the most valuable target in the chain because it touches everything, as SolarWinds showed. Run builds on ephemeral runners that are created for one job and destroyed afterwards, so that an attacker cannot persist between builds. Make builds hermetic, with all inputs declared and fetched by hash and no network access during compilation, so that undeclared inputs cannot slip in. Keep long-lived secrets out of CI by using OIDC federation to obtain short-lived cloud credentials for each job, and give each pipeline the least privilege it needs, so that a pull request from a fork cannot reach the deployment credentials. Protect the release path with branch protection and two-person review. Finally, reproducible builds, where the same source always produces bit-for-bit identical output, let independent parties rebuild and compare; the Reproducible Builds project and distributions such as Debian have made most of their packages reproducible, and a mismatch is strong evidence of tampering.
The EU Cyber Resilience Act
The Cyber Resilience Act (CRA), Regulation (EU) 2024/2847, sets horizontal cybersecurity requirements for products with digital elements placed on the EU market: hardware and software products, including their remote data processing solutions. It entered into force on 10 December 2024 and applies in stages. The rules on notifying conformity assessment bodies apply from 11 June 2026. The reporting obligations in Article 14 apply from 11 September 2026: a manufacturer must report an actively exploited vulnerability or a severe incident affecting the security of its product with an early warning within 24 hours of becoming aware, a fuller notification within 72 hours and a final report later (for a vulnerability, within 14 days after a corrective measure is available; for a severe incident, within one month), through the single reporting platform run by ENISA, the EU cybersecurity agency. Most other obligations apply from 11 December 2027.
The essential requirements in Annex I are architectural as much as procedural: products must be designed with an appropriate level of security, delivered with a secure default configuration, limit attack surfaces, protect the confidentiality and integrity of data, minimise data, and be able to log security events and receive security updates. Manufacturers must identify and document components, including by drawing up an SBOM covering at least top-level dependencies, handle vulnerabilities through a coordinated disclosure policy, and provide security updates for a support period that should normally be at least five years unless the product is expected to be used for less. Fines for breaching the essential requirements or the manufacturers' obligations can reach EUR 15 million or 2.5% of worldwide annual turnover, whichever is higher.
Scope matters. Free and open-source software developed or supplied outside a commercial activity is outside the CRA, and a lighter regime applies to open-source software stewards that support such software for commercial use. Pure software as a service is generally outside the CRA unless it is the remote data processing solution of a product; cloud and other digital service providers may instead fall under the NIS2 Directive. Whether a particular product is in scope, and which conformity route applies, is a legal question for each organisation; the architectural consequence is that SBOMs, vulnerability handling, update mechanisms and security logging become evidence that has to exist, not good intentions.
Checklist: supply-chain security for a service team
Most of these are platform features once and habits afterwards.
- Keep an inventoryGenerate an SBOM for every build and store it with the artefact, so that "are we affected?" is a query, not a project.
- Lock and verify dependenciesCommit lockfiles, install with hash checking in CI, and fail the build if the lockfile and manifest disagree.
- Choose dependencies deliberatelyBefore adding one, check maintenance, number of maintainers, OpenSSF Scorecard results, licence and how much code it pulls in transitively.
- Update continuously, but not instantlyLet a bot propose small updates every week; apply a short minimum release age for routine updates; fast-track security fixes.
- Scan, then triageRun software composition analysis and image scanning in CI and continuously in the registry; prioritise by exploitability and reachability, not by count.
- Pin build tools as well as librariesPin CI actions and plug-ins to full commit hashes and base images to digests; review changes to them like code.
- Isolate and harden the buildEphemeral runners, no long-lived secrets (use OIDC federation), least-privilege tokens, protected branches and two-person review for release paths.
- Sign and record provenanceSign artefacts with Sigstore or an equivalent, publish SLSA provenance, and aim for Build L3 on the platform.
- Verify at deploymentAdmission policies reject images that are unsigned, signed by the wrong identity or lack provenance from the expected repository.
- Plan for the next Log4ShellRehearse finding and patching one library across every service within a day, and know your CRA, contractual and customer reporting duties.
Questions people ask
Should we vendor (copy) all dependencies into our repository?
Vendoring gives control and offline builds but makes updates harder and hides upstream fixes. A lockfile with hashes plus a private registry mirror that caches approved packages gives most of the control with less maintenance.
Is a private registry enough to stop dependency confusion?
Only if builds are configured to resolve internal names exclusively from it. Use scoped or namespaced package names, register your internal names on public registries, and configure the package manager so that a public package can never override an internal one.
Who in the organisation owns supply-chain security?
Usually shared: a platform or security team provides the hardened pipeline, signing, scanning and policies once; each service team keeps its dependencies few, current and justified.
Check your understanding
Why did SolarWinds customers' signature checks not protect them?
The backdoor was inserted during SolarWinds' own build, so the trojanised update was genuinely signed by SolarWinds. A signature shows who produced an artefact, not that the build was clean.
What does a lockfile add to a manifest?
The manifest states acceptable version ranges; the lockfile records the exact versions resolved and usually their hashes, so every build installs the same bytes and a changed package fails verification.
Why is referring to a CI action by tag riskier than by commit hash?
A tag can be moved by whoever controls the repository, including an attacker, so the same reference can silently run different code; a full commit hash identifies immutable content.
What question does an SBOM answer that a vulnerability scanner alone does not answer quickly across an estate?
Which deployed artefacts, in which services and environments, contain a given component and version, so that exposure to a newly announced vulnerability can be found everywhere at once.
Sources and further reading
Have a question about this section? ArchitectBot, the course’s AI study assistant, answers from the course first.
Ask: “How should our team prioritise hundreds of dependency vulnerability alerts, and which ones actually matter?”