Architecture across the lifecycle: up-front, emergent and continuous architecture

The old argument: up front or emergent?

For most of software's history, architecture was something you did at the start. In a sequential ("waterfall") process, requirements were gathered, then an architecture and detailed design were produced and approved, then the system was built to that design. This is big design up front (BDUF). Its logic is sound in some settings: when requirements are stable and well understood, when changes after delivery are extremely expensive (flight software, medical devices, hardware-bound systems), or when many contractors must build parts that fit together, a thorough design before construction reduces risk.

Its weaknesses are equally well known. Requirements change, and an elaborate design made early encodes assumptions that later prove wrong. Feedback from real use arrives only at the end, when change is most expensive. Teams design for imagined needs ("we might need to support five currencies") and pay for flexibility nobody uses. And documents that took months to approve tend to be followed after they have stopped being true.

Agile methods reacted against this. Extreme Programming in particular argued for emergent design: build the simplest thing that works for the features in hand, keep the code clean with continuous refactoring and automated tests, and let the design grow as the system does. Kent Beck's rules of simple design and the principle "you aren't gonna need it" (YAGNI) express the same idea. Emergent design works well inside a module, and for small systems where every decision is a two-way door. Its weakness appears at scale: some decisions (the data model of a system of record, the boundaries between teams, the security model, the choice between synchronous and asynchronous integration) are expensive to reverse and shape everything built on top. Leaving them to emerge often means they are made by accident by whoever writes the first code.

What the Agile Manifesto does and does not say

One of the twelve principles behind the Agile Manifesto (2001) reads: The best architectures, requirements, and designs emerge from self-organizing teams. It is frequently quoted as if it meant that agile teams do not need architecture or architects. It does not say that.

Read carefully, the principle makes a claim about who produces the best architectures (the people who build the system, organised by themselves rather than directed from outside) and how (they emerge through the work, rather than being handed down complete). It says nothing about architecture being unimportant, and the same document contains another principle: continuous attention to technical excellence and good design enhances agility. A team that ignores structural decisions is not following the manifesto; it is ignoring half of it.

What the principle does rule out is an architecture imposed from an ivory tower by people who will not live with its consequences, and a phase in which the architecture is finished and frozen before anyone writes code. In practice, the most common reconciliation is that architecture becomes a continuous activity, shared within the team, with an architect (or architecturally minded engineers) acting as guide, facilitator and keeper of the big picture rather than as a gatekeeper.

How much architecture? The risk-driven model

If neither extreme is right, how much architecture work is enough? George Fairbanks gave one of the clearest answers in Just Enough Software Architecture: A Risk-Driven Approach (2010). His risk-driven model says the effort you spend on architecture should be proportionate to the risk of failure. It has three steps, repeated as needed: identify and prioritise risks (ideally as concrete failure scenarios, such as "the booking page times out when 5,000 patients log in at 8 a.m."); select and apply techniques that reduce those risks (a model, a prototype, a load test, an analysis, a design pattern); and evaluate the risk reduction, stopping when the remaining risk is acceptable.

The model is powerful because it gives a stopping rule. A simple internal tool built on familiar technology with few users faces few architectural risks, and a sketch on a whiteboard may be enough. A payment platform handling millions of transactions, subject to regulation, faces many, and deserves careful modelling, prototyping and review. The same team may apply very different amounts of architecture to different parts of one system. Fairbanks also stresses that time spent designing is time not spent building or testing, so over-designing is a real cost, not a safe default.

Intentional architecture and the architectural runway

Large organisations face a coordination problem that a single team does not: dozens of teams must build parts that fit together. The Scaled Agile Framework (SAFe), a widely adopted and also widely debated framework for scaling agile practices, describes architecture as a balance between emergent design within teams and intentional architecture: deliberate, cross-team planning of the structures and guidelines that teams share. It calls the result the architectural runway: the existing code, components and technical infrastructure needed to implement near-term features without excessive redesign and delay. Work to extend the runway (for example, introducing a messaging platform before the features that need it) is planned explicitly alongside feature work, and features consume the runway as they use it.

Described neutrally, the runway metaphor captures something true for any organisation: if the infrastructure a feature needs does not yet exist, that feature will be slow, and building the infrastructure takes lead time. Critics of SAFe argue that it can reintroduce heavyweight, top-down planning under agile vocabulary; supporters argue that some central coordination is unavoidable at scale. You do not need to adopt SAFe to use the idea: "what foundations will next quarter's features need, and are we building them in time?" is a good question for any architect.

Continuous architecture

Murat Erder and Pierre Pureur, in Continuous Architecture: Sustainable Architecture in an Agile and Cloud-Centric World (2015) and Continuous Architecture in Practice (2021, with Eoin Woods), describe architecture as an ongoing flow of decisions rather than a phase or a document. Their six principles are: architect products, not just solutions for projects; focus on quality attributes, not on functional requirements; delay design decisions until they are absolutely necessary; architect for change by leveraging the power of small, loosely coupled parts; architect for build, test, deploy and operate; and model the organisation of your teams after the design of the system you are working on.

Two of those principles echo earlier sections of this chapter: delaying decisions is the last responsible moment, and modelling teams after the system is a deliberate use of Conway's law. The practical emphasis is on the unit of architectural work being the decision, recorded as it is made, and on feedback loops (automated tests, fitness functions, production monitoring) that tell the team whether the architecture still meets its quality goals.

Walking skeletons and spikes

Two simple techniques bring architecture and delivery together from the first week. A walking skeleton, a term associated with Alistair Cockburn, is a tiny implementation of the system that performs one small end-to-end function through all the main architectural components. For a hospital booking system, it might let a test user log in, see one hard-coded clinic's free slots fetched from the patient administration system, book one, and receive a text message, deployed to a production-like environment through the real delivery pipeline. It does almost nothing useful, but it proves that the pieces connect, that the pipeline works, and that the architecture can be built. Every later feature adds flesh to a skeleton that already walks.

A spike (another term from Extreme Programming) is a short, time-boxed experiment to answer one question or reduce one risk: can the PAS interface handle 50 requests a second? How long does the identity provider take to issue a token? Can our chosen library generate accessible PDFs? Spikes are usually thrown away; their output is knowledge, recorded in a decision. They are the cheapest way to move a decision from guesswork to evidence before its last responsible moment.

Architecture activities in an iterative cycle

In an iterative process, architecture work does not happen once; it recurs in every cycle, at a depth that matches the risks in front of the team. The iSAQB curriculum describes the architect's core tasks in a way that maps naturally onto such a cycle, and The architect's role introduced them. Here they are as a loop:

Clarify requirements: find the stakeholders and concerns, sharpen the quality goals and constraints, and identify the risks (see Stakeholders and concerns). Design structures: make or revise the significant decisions, choosing styles and patterns and defining boundaries and interfaces. Communicate and agree: explain the decisions to the people who build, run and pay for the system, in views suited to each audience, and record them. Accompany implementation: work with the teams as they build, answer questions, review code and adjust the design when reality pushes back. Evaluate: check that the system meets its quality goals, through reviews, tests, measurements in production and automated fitness functions. Evolve: act on what evaluation found, repay debt, and adapt to changed requirements, which brings you back to clarifying requirements.

Architecture in DevOps and continuous delivery

Continuous delivery, in which every change is built, tested and kept releasable, and DevOps, which joins development and operations responsibility, change both the goals of architecture and its feedback loops.

They change the goals because deployability, testability and operability become first-class quality attributes. A system cannot be released many times a day if components must be deployed together in a fixed order, if tests need a full copy of production, or if nobody can tell from the monitoring whether a release made things worse. The research summarised in Accelerate (Nicole Forsgren, Jez Humble and Gene Kim, 2018) found that a loosely coupled architecture, one that lets teams test and deploy their parts independently, was among the strongest predictors of continuous delivery performance. Architecture is therefore not just helped by good delivery practice; it is one of its preconditions.

They change the feedback loops because the running system becomes the most honest architecture review. Production metrics, traces and incident reviews show where the architecture is weak far more reliably than diagrams do. Architectural rules can be enforced as automated fitness functions in the pipeline (Chapter 11), infrastructure is defined as code and reviewed like code (Chapter 10), and techniques such as feature flags and canary releases make many decisions cheaply reversible. The architect's role shifts accordingly: less approval of documents, more shaping of the platform, pipeline and guardrails that let teams make good decisions quickly.