Lightweight reviews and risk-storming

Why heavyweight review boards often fail

Many organisations govern architecture through an architecture review board: a panel of senior architects that meets monthly, receives a slide deck or a long document, and approves or rejects designs. The intention is sound: independent, experienced eyes on significant decisions. The results are often poor, for reasons that are structural rather than personal.

Timing is the first problem. A board that meets at fixed points sees a design either too early (nothing is decided yet) or, far more often, too late, when the team has already built a prototype, signed up to a vendor and promised a date. Rejection is then so costly that the board approves with conditions nobody tracks. Context is the second: reviewers who spend an hour on a system they have never seen ask generic questions and miss the specific risks that the team's own developers, operations staff and security engineers could name in minutes. Incentives are the third: when a review is an exam, teams present the design they want approved, not the doubts they have, and the board learns little. Queueing is the fourth: a monthly meeting with a full agenda adds weeks of waiting to every significant decision, which pushes teams to avoid the board by labelling decisions as minor.

None of this means independent review is worthless. Regulated industries (banking, healthcare, aviation) often need a documented sign-off, and some decisions really do need the authority of a senior group. The lesson is that review should be frequent, early, collaborative and focused on risks, with formal approval reserved for the few decisions that genuinely require it. Section 11.4 returns to how governance can work without slow gates.

What a good lightweight review gives you

A lightweight review is a short, structured session, typically an hour or two, in which the people who know a design best and a few people who see it fresh look for risks together. It aims at four things: discovering risks while they are cheap to address, building a shared understanding of the design across the team, giving an independent challenge to the architect's assumptions, and leaving a record of the risks accepted, mitigated or rejected. Its output is a prioritised risk list with owners, not a verdict.

Several techniques fit this description. They differ in what they examine (a diagram, a plan, a document or a running system) and in what they are best at, and they combine well.

Risk-storming

Risk-storming is a visual, collaborative technique created by Simon Brown, the author of the C4 model (Section 4.3). It rests on a simple observation: different people see different risks, and the best way to collect them is to put everyone in front of the same diagrams and ask them to look independently before discussing. It has four steps.

1. Draw the architecture. Put up diagrams of what you plan to build or change, ideally at more than one level: a system context diagram and a container diagram are usually enough, with a component diagram for the riskiest container. 2. Identify risks individually. Each participant silently writes one risk per sticky note, with a short description and a priority, for about ten minutes. Silence matters: it prevents the first or most senior speaker from anchoring everyone else, which is the main way group discussions lose information. 3. Converge the risks on the diagrams. Everyone places their notes on the part of the diagram the risk concerns. Clusters appear immediately, and the diagram becomes a heat map. 4. Review and summarise. The group discusses the clusters, then pays special attention to two kinds of note: risks raised by only one person (often someone with knowledge others lack, such as the support engineer) and risks given very different priorities by different people (a sign of a misunderstanding). Agreed risks go into a risk register with owners; items agreed not to be risks are recorded too.

Priority is usually scored as probability times impact, each rated low (1), medium (2) or high (3), giving scores from 1 to 9. Brown's guidance bands them as high (6 or 9), medium (3 or 4) and low (1 or 2), conventionally shown with pink or red, yellow or amber, and green notes. A prompt list helps people think beyond their own speciality: scalability, availability, performance, security, data consistency, third-party dependencies, cost, technology maturity, team skills, and delivery and operation.

Pre-mortems

Risk-storming looks at a structure. A pre-mortem looks at a plan. Gary Klein described it in the Harvard Business Review in September 2007. A post-mortem explains why a patient died, which helps everyone except the patient; a pre-mortem is held before the project starts, when the answers can still change its course.

The procedure is short. The leader briefs the group on the plan or decision, then announces that it is now some months in the future and the effort has failed, badly. Each person spends a few minutes writing down every reason they can think of for the failure, independently. The leader then goes round the room collecting one reason at a time until the lists are exhausted, and the owner of the plan reviews them for ways to strengthen it. Klein cites research by Deborah Mitchell, Jay Russo and Nancy Pennington (1989) finding that imagining an event has already happened (prospective hindsight) increases people's ability to identify reasons for future outcomes by about 30%. The framing also changes the social dynamics: the question is no longer whether the plan might fail, which sounds disloyal, but why it did, which invites creativity.

For architecture, pre-mortems are particularly useful before large commitments: adopting an event-driven style, migrating to a new cloud, replacing a legacy system (Section 11.6) or introducing a platform. Typical outputs that a diagram-based review would miss include the team lacking skills in the new technology, the migration stalling halfway with two systems to run, a key supplier changing its pricing, and the business losing patience before benefits appear.

Architecture katas

Architectural katas were created by Ted Neward, inspired by Dave Thomas's code katas, as practice exercises for architects. A kata is a short brief for a fictional system (a few paragraphs of requirements, users and constraints, deliberately incomplete). Small teams of three to five people work on it for a fixed time, questioning a moderator who plays the customer to uncover missing requirements, then sketch an architecture and present it to the other teams, who challenge it hard but fairly before the room votes. Neal Ford maintains a public collection of katas, and O'Reilly has run architectural kata competitions.

Katas are not reviews of a real design, but they contribute to review culture in two ways. They build the skill of explaining and defending a design under questioning, and of asking good questions about someone else's, which is exactly what lightweight reviews need. And an internal kata on a real upcoming problem, with two or three teams designing independently, is a cheap way to explore the option space before committing: when three teams produce three different designs, the differences show where the real trade-offs lie.

TARA and checklist-based reviews

Reviews of systems that already exist face a different problem: the architecture as built may differ from the architecture as described, and the evidence is spread across people, documents, code and production data. Eoin Woods, co-author of Software Systems Architecture, described the Tiny Architectural Review Approach (TARA) in 2011 as a deliberately lightweight, industrial alternative to scenario-heavy methods. A small number of reviewers elicit the system's key requirements and context, establish its actual structure, assess its implementation using automated analysis (module dependencies, size, code metrics, test coverage) alongside runtime information such as logs, use expert judgement to link what they find to the requirements, and present findings and recommendations in a predefined form. Its strength is speed and evidence; its weakness, which Woods acknowledged, is dependence on the reviewers' expertise.

Checklists complement every technique. A checklist does not find risks by itself, but it stops a review from forgetting a whole category (backup restore, secrets, cost at scale) because nobody in the room happened to think of it. Keep checklists short, specific to your organisation's history of incidents, and phrased as questions that demand evidence rather than yes or no answers.

Design documents and RFC reviews

Section 4.8 described design documents and RFCs as a way to propose a change in writing before building it. Reviewing them well is a skill of its own. A written review has real advantages over a meeting: reviewers comment when they have time to think, remote and quieter colleagues contribute on equal terms, and the comments become a record. Its typical failures are documents nobody reads, comment threads that never close and reviews that become an unofficial gate.

Good practice is to state the decision sought and the deadline for comments at the top, name the reviewers whose input is required (security, operations, the owners of affected services) and invite anyone else; ask reviewers to label each comment as a question, a suggestion or a blocking concern, so the author knows which ones must be resolved; review against the quality scenarios and the checklist below, not against personal taste; close with a short meeting only for unresolved concerns; and record the significant decisions as ADRs. In organisations using the advice process (Section 11.4), the RFC is the vehicle for seeking advice, and the decision stays with its author.