Requirements, version control, testing, delivery and the journey of one request

From an idea to a requirement

Software starts with a need: a restaurant wants to take orders online, a hospital wants patients to book appointments themselves. Turning that need into something a team can build is the work of requirements. Some requirements describe what the system must do ("a customer can pay by card"); others describe how well it must do it ("checkout completes within two seconds for 95% of customers"). Chapter 2 shows that the second kind, the quality attributes, drive most architectural decisions.

Many teams capture functional needs as user stories: short descriptions written from the user's point of view. Kent Beck introduced them in Extreme Programming in the late 1990s, and a team at the London company Connextra devised the template most people now use in 2001:

As a hungry customer,
I want to reorder a meal I have ordered before,
so that I can eat sooner without searching the menu again.

Acceptance criteria
- My last 10 orders appear on the home screen.
- Tapping "Reorder" fills the basket with the same items.
- Items that are no longer available are flagged, not silently dropped.

A story is deliberately small and incomplete. Ron Jeffries described it as a card, a conversation and a confirmation: the card is a reminder, the details are worked out in conversation between the people who want the feature and the people building it, and the acceptance criteria confirm when it is done. Practitioners disagree about how much should be written down in advance. Teams in regulated industries such as medical devices and aviation must keep formal, traceable requirements; product teams often prefer lighter stories refined just before work starts. Both are legitimate in their context.

Version control with Git

Source code is the team's most valuable shared document, and many people change it every day. Version control records every change, who made it and why, lets people work in parallel without overwriting each other, and allows any earlier state to be restored. Almost every team now uses Git, created by Linus Torvalds in 2005 for the development of the Linux kernel, usually together with a hosting service such as GitHub, GitLab or Bitbucket.

A few ideas cover most daily use. A commit is a saved snapshot of a set of changes, with a message explaining it ("Add tip button to checkout"). A branch is a separate line of work: a developer creates a branch from main, makes commits there without disturbing anyone, and when finished merges the branch back, combining the two histories. If two people changed the same lines, Git reports a merge conflict for a human to resolve.

On hosting services, merging usually goes through a pull request (GitLab calls it a merge request): a web page showing the proposed change, where automated checks run and colleagues carry out code review, reading the change, asking questions and suggesting improvements before approving it. Review catches defects, but its larger benefits are spreading knowledge (at least two people understand every change) and keeping the design consistent. Architects often take part in reviews of changes that affect structure or interfaces.

git switch -c feature/tip-button     # create a branch and move onto it
# ... edit files ...
git add checkout.py test_checkout.py
git commit -m "Add tip button to checkout"
git push -u origin feature/tip-button  # upload; then open a pull request
# after review and green checks, the branch is merged into main

Teams differ on branching. Some keep long-lived branches per feature or release; others practise trunk-based development, merging small changes into main at least daily and hiding unfinished features behind switches called feature flags. Research by the DORA programme associates short-lived branches with better delivery performance, but the right choice also depends on how a team releases.

Testing: the pyramid

Every change risks breaking something that used to work, which is called a regression. Automated tests are programs that check other programs, so they can be run on every change in minutes instead of by people over days. They come in layers. A unit test checks one small piece, such as a function, in isolation, and runs in milliseconds. An integration test checks that parts work together: the order code with a real database, or the shop with a payment provider's test environment. An end-to-end test drives the whole system as a user would, for example tapping through the app from menu to order confirmation.

Mike Cohn's testing pyramid, published in his 2009 book Succeeding with Agile, suggests the proportions: many unit tests, fewer integration tests, and few end-to-end tests. The higher layers are more realistic but slower, more expensive to maintain and more prone to flaky failures that come and go for reasons unrelated to the change. Some practitioners argue for a different shape, such as more integration tests for systems that are mostly glue between services; the principle all agree on is to catch most problems with fast, reliable, focused tests.

def delivery_fee(basket_total, distance_km):
    if basket_total >= 25:
        return 0.0                      # free delivery on larger orders
    return round(1.50 + 0.40 * distance_km, 2)

def test_free_delivery_over_threshold():
    assert delivery_fee(30.00, 5) == 0.0

def test_fee_grows_with_distance():
    assert delivery_fee(10.00, 5) == 3.50

Environments: development to production

Code passes through several environments on its way to customers, each a separate copy of the system with its own servers, data and settings. The aim is to find problems where they are cheap and harmless, before they reach production, the live environment real users depend on.

The more environments differ from production, the more problems slip through: a test database with a hundred rows will not reveal the missing index that makes a query take forty seconds against ten million. Keeping environments alike is easier when they are built from the same container images and the same scripted configuration, a theme of infrastructure as code.

Continuous integration, delivery and deployment

Continuous integration (CI) is the practice of merging everyone's work into the shared main line frequently, at least daily, with an automated build and test run checking every change. It grew out of Extreme Programming in the 1990s. Its point is that integration problems are found within minutes, while the change is fresh in the author's mind, instead of in a painful "integration phase" weeks later. A team practising CI treats a failing build as an emergency: fixing it comes before any new work.

Continuous delivery (CD) extends CI so that the software is always releasable: every change that passes the pipeline could be put into production at the press of a button, and releasing becomes a routine business decision rather than a risky event. The approach was set out by Jez Humble and David Farley in their 2010 book Continuous Delivery. Continuous deployment goes one step further and releases every passing change automatically, with no human approval; teams that do this may deploy many times a day.

The automation that carries a change from commit to production is a pipeline. Because each step is automated and identical every time, releases stop depending on one person remembering a checklist at midnight.

Ways of working: waterfall, Scrum and Kanban

Teams also choose how to organise the work itself. In a sequential approach, often called waterfall, the project moves through phases in order: requirements, design, implementation, testing, deployment. Each phase finishes before the next begins. The name is usually traced to Winston Royce's 1970 paper, although Royce did not use the word and in fact warned that a single pass through the phases was risky. Sequential plans still suit work where requirements are fixed and change is very costly, such as some regulated or hardware-dependent projects. Their weakness for most software is that users see nothing until late, so misunderstandings surface when they are most expensive to fix.

Agile approaches, named in the Agile Manifesto of 2001, deliver working software in short cycles and adjust plans as they learn. Scrum is the most widely used framework: a small team works in fixed cycles called sprints, of one month or less, each starting with planning and ending with a review of the work and a retrospective on how the team worked. The 2020 Scrum Guide defines three accountabilities: the Product Owner (deciding what is most valuable), the Scrum Master (helping the team use Scrum effectively) and the Developers (doing the work). Kanban, adapted for software from Toyota's production system, has no fixed cycles: work items flow across a visible board (to do, doing, review, done), and the team limits how many items may be in progress at once so that work is finished before more is started.

None of these is right for every team, and many organisations mix them: Scrum for product teams, Kanban for support and operations, a more sequential plan around a fixed regulatory deadline. Calling a process "agile" does not make it so; what matters is whether working software reaches real users often enough to learn from them.

DevOps: one team's responsibility from code to customer

Traditionally, developers wrote software and a separate operations team ran it, with conflicting incentives: developers were rewarded for change, operations for stability. DevOps is the movement, named at the first DevOpsDays conference in Ghent in 2009, that brings the two together. It is primarily a culture of shared responsibility ("you build it, you run it"), supported by practices such as CI/CD, infrastructure as code, monitoring and blameless review of incidents. It is not a job title or a tool, although both are sold under the name. The DORA research programme is best known for measuring delivery performance with four key metrics: how often a team deploys, how long a change takes to reach production, how often changes fail, and how quickly service is restored after a failure.

Worked example: the journey of one tap

Everything in this chapter comes together in a single moment. A customer has filled a basket in a food-delivery app and taps Place order. Within a second or two the screen says "Order confirmed", and twenty minutes later the phone buzzes: "Your rider is on the way". Here is what happens, step by step, with the concept behind each step and where it was introduced.

Why this matters for architecture

The journey of one tap uses a dozen components, three organisations and both synchronous and asynchronous communication, and it still had to be designed: someone decided that payment is synchronous and notifications are not, that orders live in a relational database, that the service runs in containers across two zones, that the payment provider is isolated behind one module. Deciding how these parts are arranged, which qualities each arrangement achieves and what it costs, and recording why, is software architecture. The way software is built matters to architecture too: an architecture that cannot be tested in pieces, or released without coordinating ten teams, slows every change. Chapter 1 begins with what software architecture is.