Be Wise Apps
Leer en español · Lire en français · Auf Deutsch lesen
Section 2.8 of 2.8 · Intermediate · 22 min read
Eliciting, prioritising and trading off quality attributes
Video lesson
5 min · narrated, with captionsListen only (audio)
Read the transcript
In this final lesson of the chapter, you'll learn how to find the qualities that matter, rank them, and decide what to give up.
Planning a family holiday, everyone wants something different. One wants it cheap, another wants a pool, another wants short travel times. You can't have everything. So you sit down together, list what each person really needs, vote on what matters most and agree what to give up. Architects do the same with a system's qualities.
Quality requirements rarely arrive in a backlog. People assume them, so you have to draw them out. Start with stakeholders: customers care about speed, operations about recovery, finance about run cost. Business goals imply qualities. Expanding into three new countries implies localisation, data residency and scalability. Regulation creates constraints that can't be negotiated. Every past outage or breach reveals a quality that was under-specified. And if rivals load in one second, two seconds is a quality problem. Finally, measure the existing system. Its actual load and latency give a baseline that turns faster into a number.
Qualities emerge best when stakeholders hear one another. That's the idea behind the Quality Attribute Workshop, from the Software Engineering Institute. It's usually a day, with five to thirty people, held before the architecture is fixed. The first four steps set the context: introductions, the business goals, the current architecture plans, and the key drivers heard so far. Then the scenario work. Everyone proposes scenarios in turn, and near-duplicates are merged, but only when their proposers agree. Next comes voting. Each person gets votes equal to about thirty per cent of the number of scenarios. Scarce votes expose real priorities. Finally, the top four or five scenarios are written out in full, with all six parts.
To organise the results, architects use a utility tree. The root is utility: the overall goodness of the system. It branches into quality attributes, then into specific concerns, such as order latency or checkout uptime. The leaves are concrete scenarios, each rated for business importance and technical difficulty: high, medium or low. Scenarios rated high and high matter most and are hardest. Here, one is placing an order at the peak rate. The other is surviving a payment provider outage. These two get the earliest design attention. This modifiability scenario is medium importance and low difficulty, so ordinary good design will do.
Let's see it in practice. A two-hour workshop for a food-delivery platform produced twenty six scenarios. With twelve participants, each got eight votes: thirty per cent of twenty six is seven point eight, rounded up. One leading scenario: at the Friday peak of three thousand orders a minute, ninety five per cent of confirmations return within one and a half seconds, and ninety nine per cent within three. A quick capacity check. Three thousand a minute is fifty a second. The business plans to double volume, so a hundred a second. Using Little's law, even if every order took the full one and a half seconds, at most about a hundred and fifty would be in progress at once. The second high, high scenario: when the payment provider times out, orders are queued, none is lost, and customers are told within ten seconds. Adding a new restaurant point of sale vendor is medium importance and low difficulty. So the team prototypes and load-tests the order path and the queued payment design first.
Qualities often conflict, because a tactic that helps one hurts another. Encryption and checks add work to every request. During a network failure, a replicated system must choose between answering with possibly stale data, and refusing. A courier's position can be slightly stale. A payment can't. Layers and interfaces help change, but add hops. Redundancy and extra regions cost money, so buy nines only where an outage costs more than preventing it. And strong authentication adds friction. The right answer always depends on context: which business goal each side serves, and what each option costs.
A trade-off that isn't written down will be argued again. Record significant ones in architecture decision records, including what you've consciously given up. Make each target measurable. For order latency: the scale is seconds at the ninety fifth percentile, the meter is real-user monitoring at the Friday peak. Fail above three seconds, goal one and a half, stretch one. Then check it automatically. A fitness function is an automated check, such as a load test that fails the build if latency drifts. That's where this chapter ends: a vague wish becomes a scenario, gets a priority, and becomes a check.
The full section has every workshop step, the complete utility tree example, a table of conflicts and a checklist for running your own workshop. That completes Chapter 2. When you're ready, try the optional chapter test to check your understanding. Well done.
What you need to know
- Quality requirements must be elicited from stakeholders, business goals, regulation, past incidents, competitors and measurements of the existing system, because they are rarely stated unprompted.
- The SEI Quality Attribute Workshop has eight steps; its outputs are architectural drivers and prioritised, refined scenarios, ranked by voting with a fixed vote budget.
- A utility tree refines utility into quality attributes, refinements and concrete scenarios, each rated (business importance, technical difficulty); (H,H) scenarios get the earliest design attention.
- Qualities conflict (performance and security, availability and consistency, modifiability and performance, cost and almost everything, usability and security), and the right resolution depends on context and business value.
- Record significant trade-offs in decision records and make each target measurable with a scale, a meter and levels, then track it over time.
- Fitness functions turn scenario response measures into automated checks that run as the system evolves.
Where quality requirements come from
Functional requirements usually arrive in a backlog. Quality requirements rarely do: stakeholders assume them ("of course it must be fast and secure") or do not know they need to state them. Architects therefore have to elicit them actively, from several sources.
Stakeholders. Each group has different concerns: customers care about speed and ease of use, operations about monitoring and recovery, finance about run cost, support about diagnosing problems, compliance about audit trails. Chapter 1's stakeholder map is the starting point. Business goals. "Expand into three new countries next year" implies localisation, data residency and scalability; "cut customer support costs by 20%" implies usability and self-service. Regulation and contracts. Data protection, accessibility law, sector rules and customer SLAs create non-negotiable constraints. Incidents. Every past outage, breach or slow day reveals a quality that was under-specified; post-incident reviews are a rich source. Competitors and the market. If rivals load in one second, two seconds is a quality problem. The existing system. Its measured behaviour today (actual load, actual latency percentiles, actual change lead time) gives a baseline that turns "faster" into a number.
Interviews and questionnaires work, but quality attributes emerge best when stakeholders hear one another. A product owner who learns from the operations lead what an hour of downtime at Friday dinner time costs will prioritise differently. That is the idea behind structured workshops.
The Quality Attribute Workshop
The Quality Attribute Workshop (QAW), developed at the Software Engineering Institute (Barbacci and colleagues; the third edition of the method report appeared in 2003), is a facilitated session, typically a day long, held early, before the architecture is fixed. It brings 5 to 30 stakeholders together to produce prioritised, refined quality attribute scenarios. It has eight steps.
The eight QAW steps
What happens at each step
- 1. Presentation and introductionsFacilitators explain the purpose and method; participants introduce themselves and their concerns.
- 2. Business or mission presentationA business representative describes the context, goals and high-level requirements: why the system exists and what matters most.
- 3. Architectural plan presentationA technical lead presents current plans, existing systems, known constraints and technology commitments, however early.
- 4. Identification of architectural driversFacilitators distil the key architectural drivers heard so far (goals, qualities, constraints) and check them with the group.
- 5. Scenario brainstormingParticipants propose scenarios in round-robin turns; facilitators make sure every driver has at least one, and that normal use, growth and exploratory (stress) scenarios are all represented.
- 6. Scenario consolidationNear-duplicates are merged, but only when their proposers agree, so that votes are not split.
- 7. Scenario prioritisationEach participant receives a vote budget of about 30% of the number of scenarios and spends it in two rounds, any number of votes per scenario. The tally ranks the scenarios.
- 8. Scenario refinementThe top four or five scenarios are written out in full: source, stimulus, artifact, environment, response and response measure, with the business goals they serve and open questions.
Teams that cannot spare a full day often run a shorter "mini-QAW", a lighter variant described by practitioners such as Michael Keeling, Will Chaparro and Thijmen de Gooijer, using a quality attribute taxonomy as a prompt and a few hours with a smaller group. The essentials survive: stakeholders together, scenarios not adjectives, explicit voting, and refinement of the winners.
The utility tree
A utility tree is a way to organise and prioritise quality requirements, introduced as part of the Architecture Tradeoff Analysis Method (ATAM), which Rick Kazman, Mark Klein and Paul Clements developed at the SEI; the evaluation method itself is covered in Chapter 11. The root is utility, the overall goodness of the system. The second level holds quality attributes (performance, availability and so on). The third level refines each into more specific concerns (for performance: order latency, tracking freshness). The leaves are concrete, measurable scenarios.
Each leaf is rated with a pair: business importance (how much the scenario matters to the business: High, Medium or Low) and technical difficulty or risk (how hard it is to achieve with the architecture: High, Medium or Low). The pair is written (B,T), for example (H,M). Scenarios rated (H,H) are the architecture's top priorities: they matter most and are hardest, so they deserve early design effort, prototypes and analysis. (H,L) scenarios matter but are easy, so a standard approach will do; (L,H) scenarios are candidates to drop or postpone, because they are hard but not worth much.
The tree is useful even outside a formal evaluation. It makes trade-offs visible on one page, shows gaps (a quality with no scenarios), and stops the loudest stakeholder from setting priorities alone.
A utility tree for a food-delivery platform
Worked example
Worked example: building the utility tree for a food-delivery platform
A food-delivery platform connects customers, 4,000 restaurants and 9,000 couriers in one country. The business plans to double order volume within 18 months. The Friday evening peak is 3,000 orders per minute today. A two-hour QAW produced 26 scenarios; with 12 participants, each had 8 votes (30% of 26 is 7.8, rounded up to an even 8). The top-voted scenarios became leaves of the tree.
| Performance: order latency (H,H) | Customers place orders at the peak rate of 3,000 per minute; 95% of order confirmations return within 1.5 s and 99% within 3 s. High importance: abandoned baskets are lost revenue. High difficulty: the request crosses basket, pricing, payment and restaurant services |
|---|---|
| Capacity check for that scenario | 3,000 orders/min = 50 orders/s. Doubling volume gives 100 orders/s. With Little's law, even if every order took the full 1.5 s, at most about 100 x 1.5 = 150 orders would be in progress at once (Section 2.4); the real average is lower because most orders finish faster |
| Availability: checkout uptime (H,H) | When the payment provider times out during the peak, orders are accepted and queued for payment, no confirmed order is lost, and customers are told within 10 s. Difficult because it needs asynchronous payment, idempotent retries and a user experience for "pending" |
| Availability: region failure (H,M) | If the cloud region fails, service resumes in a second region within 15 minutes (RTO) with no more than 1 minute of order data lost (RPO). Important but technically well understood |
| Security: account takeover (H,M) | A credential-stuffing attack is detected and blocked within 2 minutes; no account protected by multi-factor authentication is taken over |
| Performance: tracking freshness (M,M) | A courier's position on the customer's map is never more than 5 s old under peak load |
| Modifiability: new POS vendor (M,L) | A new restaurant point-of-sale vendor is integrated in 5 person-days by adding one adapter (Section 2.6) |
| Resulting priorities | Prototype and load-test the order path and the queued-payment design first; use standard multi-region patterns for region failure; keep the adapter interface simple |
Ratings are judgements agreed by the group, not calculations. Record who rated what and why, so the ratings can be challenged later.
The tree drives the work plan: the two (H,H) scenarios get architectural spikes in the first sprints, while the (M,L) scenario needs no special attention.
Courier tracking was rated only medium in importance, which surprised the engineers; the product owner explained that customers mostly check the estimated arrival time, not the map.
Prioritising when everything is "critical"
Every stakeholder will rate their own concern as high. Techniques that force real choices help. A fixed vote budget (as in the QAW) makes people spend scarce votes. Pairwise comparison ("if you could have only one, sub-second search or zero data loss on failover?") exposes the true order. Putting money or time next to each target ("four nines will cost an extra £9,000 a month and two months of work") changes many opinions. And distinguishing constraints (legal accessibility requirements, data residency) from preferences stops non-negotiables being traded away by accident.
Priorities are not fixed. A start-up seeking its first customers rightly ranks time to market and modifiability above five-nines availability; the same product, three years later with enterprise clients and contractual SLAs, must rerank. Revisit the utility tree when the business changes.
Conflicts between qualities
Qualities interact. Some reinforce each other: modularity helps testability, which helps deployability. Many conflict, because the tactic that helps one quality hurts another. The ATAM, described by Bass, Clements and Kazman, calls the points where an architectural decision affects several qualities trade-off points. Recognising the usual conflicts lets you raise them before they surprise anyone.
Common conflicts between qualities, and how context resolves them
| Conflict | Why they pull apart | Ways to resolve it in context |
|---|---|---|
| Performance vs security | Encryption, authentication checks, input validation and audit logging add work to every request | Measure the real overhead (often small with modern TLS and hardware); cache authorisation decisions briefly; push expensive checks to the edge; never trade away security on data that would cause serious harm if exposed |
| Availability vs consistency | During a network partition a replicated system must choose between answering (possibly with stale data) and refusing | Decide per operation: a menu or courier position can be slightly stale; a payment or stock reservation cannot. See CAP and PACELC in Chapter 6 |
| Modifiability vs performance | Layers, interfaces, intermediaries and late binding add indirection and hops | Keep the indirection where change is likely; optimise the measured hot path; accept a few deliberate shortcuts, documented in a decision record |
| Cost vs nearly everything | Redundancy, extra regions, more testing and faster hardware all cost money | Tie each quality target to its business value; use error budgets and unit cost; buy nines only where an outage costs more than preventing it |
| Usability vs security | Strong authentication, timeouts and confirmations add friction | Risk-based authentication (step up only for risky actions), passkeys and single sign-on, sensible session lengths; involve users in testing |
| Time to market vs maintainability | Shortcuts speed the first release and slow later ones | Take on technical debt deliberately, record it and schedule its repayment (Chapter 1) |
The right resolution is always contextual. "Security beats performance" is a slogan, not a decision: an extra 2 ms of TLS handshake on a public website is irrelevant, while synchronous fraud checks on every card swipe at a busy stadium may be a real problem worth an architectural answer (an approve-first, check-after model with limits). Ask: which business goal does each side serve, what does each option cost, and who must agree?
Recording decisions and tracking qualities
A trade-off that is not written down will be argued again, often by people who were not in the room. Record significant ones as architecture decision records (Chapter 4): the context, the qualities in tension, the options considered, the decision and its consequences, including what you have consciously given up.
Then make each quality target measurable and visible. Tom Gilb's Planguage offers a disciplined format: a scale (what is measured, in which unit), a meter (how it is measured), and levels such as fail (the threshold below which the requirement is not met), goal (the target) and stretch or wish (more ambitious levels). Applied to the order-latency scenario: scale, seconds from submit to confirmation at the 95th percentile; meter, real-user monitoring during the Friday peak; fail above 3 s; goal 1.5 s; stretch 1 s.
Finally, track the measures over time on a dashboard that stakeholders can see, alongside SLOs (Chapter 9), and revisit them whenever the business, the load or the team changes.
From scenarios to fitness functions
The strongest way to keep a quality requirement true is to check it automatically, every time the system changes. Neal Ford, Rebecca Parsons and Patrick Kua, in Building Evolutionary Architectures (2017; second edition with Pramod Sadalage, 2022), call such a check an architectural fitness function: an objective assessment of how well the system meets an architectural characteristic. A load test in the pipeline that fails the build if p95 order latency exceeds 1.5 s at 100 orders per second is a fitness function. So is a test that fails if any module outside payments imports the payment provider's library (protecting the modifiability scenario), a scheduled failover drill that measures recovery time, or a dependency scan that blocks known vulnerable libraries.
Fitness functions are the natural end point of the work in this chapter: a vague wish becomes a scenario (Section 2.3), the scenario gets a priority in the utility tree, and the scenario's response measure becomes an automated check. Not every quality can be automated cheaply (usability still needs people), but each one you automate stops silent erosion. Chapter 11 covers their design in depth: fitness functions and evolutionary architecture.
Checklist: running a quality attribute workshop
Adapted for a half-day or one-day session; a shorter "mini-QAW" can cover the same ground with a smaller group.
- Invite the right mix5 to 30 people: business owners, users or their representatives, operations, security, support, developers, and anyone who signs off compliance. Missing operations is the most common gap.
- Send a short pre-readThe business goals, known constraints and a draft list of quality attributes (ISO/IEC 25010 as a prompt), so people arrive thinking.
- Open with business goals, not technologyA senior stakeholder explains why the system exists, what success looks like and what failure would cost.
- Agree the architectural driversA short list of the goals, qualities and constraints that will shape the architecture, checked with the group before scenarios start.
- Brainstorm scenarios in roundsEveryone contributes at least two, in turn, so quiet voices are heard. Include normal use, growth ("in two years") and stress ("when the payment provider is down").
- Merge duplicates only with consentConsolidation keeps votes from being split, but the proposer must agree that two scenarios really mean the same thing.
- Vote with a fixed budgetEach person gets votes equal to about 30% of the scenario count, spent in two rounds. This exposes real priorities and avoids "everything is critical".
- Refine the top few fullyWrite the top four or five as six-part scenarios with measurable response measures, and note the business goal each one serves.
- Record open questions and conflictsDisagreements about priority or feasibility are valuable outputs; list them with an owner.
- Publish the result within daysShare the prioritised scenarios, add them to the architecture documentation and the backlog, and plan when each will be checked.
Check your understanding
In a QAW with 20 consolidated scenarios, about how many votes does each participant get?
About 30% of 20, which is 6 votes, spent in two rounds of 3.
A utility-tree leaf is rated (L,H). What does that suggest?
It matters little to the business but is hard to achieve, so it is a candidate to drop, defer or renegotiate rather than drive the architecture.
Why is "security always beats performance" a poor way to resolve a conflict?
Because the right answer depends on the actual overhead, the harm the security control prevents and the business value of the performance; the conflict must be resolved per case with numbers and recorded.
Give one example of a fitness function for a modifiability requirement.
An automated test in the build that fails if any module other than the payment adapter imports the payment provider's library, keeping the provider replaceable.
Sources and further reading
Have a question about this section? ArchitectBot, the course’s AI study assistant, answers from the course first.
Ask: “Our product owner says every quality is critical. How can I run a short session that gets an honest ranking?”