Leer en español · Lire en français · Auf Deutsch lesen

Section 6.1 of 6.8 · Intermediate · 24 min read

Why distribute, and the fallacies of distributed computing

Q: Three services each have 99.95% availability and are called in series on every request. What is the approximate availability of the request path?

0.9995 3 ≈ 0.9985, about 99.85%, assuming independent failures.

Video lesson

5 min · narrated, with captions

Listen only (audio)

Read the transcript

Splitting a system across many machines brings real benefits, and problems that can't happen inside one program. In this lesson you'll learn when it's worth it, and how to price it before you pay.

Imagine a small restaurant where one cook does everything in one kitchen. It's simple. If the cook is ill, the restaurant is closed, but nothing gets lost between people. Now imagine a chain, with a central bakery, a sauce kitchen and many branches, all linked by delivery vans. It can serve far more customers, and one kitchen can close without closing the others. But vans get stuck in traffic and orders go missing. When a delivery doesn't arrive, nobody knows whether the bakery never got the order or the van broke down. Distributed software works like the chain.

A distributed system is a set of programs on different machines that cooperate, by sending messages over a network, to provide what users see as one service. A food-delivery app is one. The key idea of this section is that distribution is a decision with a price, not a sign of maturity.

There are a handful of good reasons to split a system. The first is scale: the courier location feed gets thousands of updates a second, the restaurant sign-up screens a few an hour. The second is fault isolation. A memory leak in recommendations can't crash checkout, because they run on different machines. Then independent releases and team autonomy. These organisational reasons are often the strongest. Others are geography and data residency, special technology needs, and security isolation, such as keeping card data in one small, hardened service. But each has a cheaper partial substitute inside a monolith, such as more copies behind a load balancer, or caches. Check the substitute first.

The defining problem is partial failure: some parts work while others have failed, are slow, or can't be reached. Here the order service asks the payment service to authorise twenty four pounds fifty, and waits. If no reply arrives before the timeout, perhaps the request was lost, and no charge was made. Perhaps the payment service crashed or stalled, and nobody knows whether the card was charged. Or perhaps the card was charged, and only the reply was lost. To the caller, all three look identical. So any operation that may be retried must be idempotent: doing it twice has the same effect as doing it once. Usually each request carries a unique key that the receiver remembers.

Engineers at Sun Microsystems listed eight false assumptions that newcomers make. The first two are that the network is reliable and latency is zero. So set a timeout on every remote call, and prefer a few coarse calls to many small ones. Bandwidth isn't infinite, and transport isn't free. Send only the fields consumers need. The network isn't secure, even inside your data centre. Authenticate every call between services. Topology changes, so use service discovery, not hard-coded addresses. And there's more than one administrator: a partner can change a rule and break your calls. Finally, the network isn't homogeneous, so use standard, versioned formats. Every one of these still causes real outages.

Every reason to distribute is paid for in the same currency. First, latency. A call inside a process takes nanoseconds. A round trip between machines in one data centre takes from a few hundred microseconds to a millisecond or more. Second, observability. A request's story is scattered across many logs, so you need correlation identifiers and distributed tracing. Third, consistency. With separate databases there's no shared transaction, so data will disagree for a while and must converge. Fourth, and most underestimated, operational load. Every service needs a pipeline, monitoring and someone on call. These costs grow with the number of deployable units.

You can price part of this on paper. Here, placing an order passes through a synchronous chain of five services. Each waits for the next, and each on its own is available ninety nine point nine per cent of the time. But availabilities multiply. Together they give only about ninety nine point five per cent: about three point six hours of downtime a month. And latencies add. With twenty milliseconds of work in each service, plus about two per hop, the median is about a hundred and ten milliseconds. This dependence on the other party being up at the same moment is called temporal coupling. So count synchronous hops, not services. Call independent services in parallel, cache stable data, and let parts that needn't answer at once work from a queue.

The full section has the complete tables and a checklist to use before you distribute. Next, in Section six point two, we look at service-based architecture and the classic service-oriented architecture. See you there.

What you need to know

A distributed system is a set of programs on different machines that cooperate by sending messages over a network to provide what users see as one service.
Legitimate reasons to distribute are scale, fault isolation, independent deployability, team autonomy, geography and data residency, different technology needs and security isolation; each has a cheaper partial substitute inside a monolith that should be considered first.
Partial failure is the defining problem: after a timeout the caller cannot tell whether the request was lost, the callee failed, or only the reply was lost, so retried operations must be idempotent.
The eight fallacies of distributed computing (Deutsch and colleagues at Sun, with Gosling's eighth) are false assumptions about reliability, latency, bandwidth, security, topology, administration, transport cost and homogeneity.
Distribution costs latency, observability, consistency and operational effort, and those costs grow roughly with the number of deployable units.
In a synchronous chain availabilities multiply and latencies add: five services at 99.9% give about 99.5%, roughly 3.6 hours of downtime a month.
Synchronous calls bind services together at run time (temporal coupling), so the number of synchronous hops in key journeys matters more than the number of services.

What makes a system distributed

A distributed system is a set of programs running on different computers that cooperate, by sending messages over a network, to provide what users see as one service. Your food-delivery app is one: the phone, the company's order and payment services, the restaurant's tablet, the courier's phone and a card processor all play a part in a single order. Chapter 5 described styles deployed as one unit (monoliths); this chapter is about styles whose parts are deployed separately and talk across the network.

Leslie Lamport, who did more than anyone to put the field on a mathematical footing, captured its essential difficulty in an internal email of May 1987: a distributed system is one in which the failure of a computer you didn't even know existed can make your own computer unusable. The joke works because it is true. Once a system depends on machines it does not control and cannot see, a whole class of failures appears that simply cannot happen inside one process.

That is why distribution is a decision with a price, not a sign of maturity. The rest of this section sets out the reasons that justify the price, the false assumptions that make people underestimate it, and a way of calculating part of it before you pay.

Why distribute at all

There are a handful of legitimate reasons to split a system across processes and machines. Each is a quality attribute from Chapter 2 that a single deployment unit struggles to deliver beyond a certain size. Each also has a cheaper partial substitute inside a monolith, and a good architect checks the substitute first.

Reasons to distribute, and what a well-built monolith can already do

Reason	What distribution offers	What a single deployment can already do
Scale	Parts with very different load can be scaled separately: the courier-location feed receives thousands of updates a second, the restaurant-onboarding screens a few an hour.	Run many identical replicas behind a load balancer, add read replicas and caches. This is enough for most systems, and far cheaper.
Fault isolation	A memory leak in recommendations cannot crash checkout, because they run in different processes on different machines.	Separate thread pools and bulkheads inside one process give partial isolation; a crash still takes everything down.
Independent deployability	One team can release its part twenty times a day without coordinating a release with anyone else.	A modular monolith with good automation can also release many times a day, but every release ships everything.
Team autonomy	Teams own a service end to end, choose their pace and are on call for it; Conway's law works for you rather than against you.	Module ownership works well up to a few teams; beyond that, a shared build and release become a queue.
Geography and data residency	Run parts near users for lower latency, or keep personal data in one jurisdiction because a law or a contract requires it.	Deploy a complete copy per region (a cell). That is itself a form of distribution, but a simple one.
Different technology needs	A recommendation model needs GPUs, search needs a search engine, a legacy integration needs a particular runtime.	Call out to the specialised component and keep everything else together.
Security isolation	Card data handled by one small, separately hardened and audited service reduces the scope of a PCI DSS assessment.	Not really: anything inside the process can reach anything else in memory.

Two observations follow from the table. First, the strongest reasons are often organisational: independent deployability and team autonomy matter most when many teams work on one product, which is why Conway's law (Chapter 3) keeps reappearing in this chapter. Second, distribution is rarely all or nothing. Many successful systems are a modular monolith plus two or three separate services, each split off for a specific, named reason.

Partial failure: the defining problem

Inside one process a function call either returns or the whole process dies with it. Across a network there is a third outcome, and it changes everything: partial failure, in which some parts of the system work while others have failed, are slow, or cannot be reached. Worse, the caller often cannot tell which.

Consider the order service asking the payment service to authorise £24.50. If no reply arrives before the timeout, the order service does not know whether the request was lost on the way, whether the payment service crashed or stalled before or after charging the card, or whether the card was charged and only the reply was lost. All three look identical from the caller's side.

Three failures that look the same

After a timeout the caller cannot know whether the work was done; only idempotent retries or a later status check can resolve it.

The practical consequences shape every distributed design. A retry might charge the customer twice, so operations that may be retried must be made safe to repeat: this property is idempotency, and it is usually achieved by giving each request a unique key that the receiver remembers. A slow dependency is often worse than a dead one, because callers keep waiting and tie up threads and connections; when one service slows down, its callers run out of resources and slow down in turn, a cascading failure. And because a network can split so that two groups of machines cannot reach each other while both keep running, data held in both places can diverge: the subject of the CAP section later in this chapter.

There is no global clock either. Each machine's clock drifts and is corrected separately, so timestamps from two services cannot be compared with complete confidence, and 'which happened first?' can be a hard question. Lamport's 1978 paper on time and clocks in distributed systems showed how to order events without relying on physical clocks; for architects the lesson is simply not to build correctness on comparing wall-clock times across machines.

If you are new to this

Step by step: when a reply never comes

This section's hardest idea is partial failure. Inside one program, a call to another part of the program either finishes or the whole program stops. Across a network there is a third possibility: you ask, and no answer comes back.

The order service asks the payment service to take £24.50 from a customer's card, and waits for the answer.
It cannot wait for ever, so it gives up after a set time limit, called a timeout: say two seconds. The two seconds pass with no reply.
Three different things could have happened, and from the order service's side they look exactly the same: the request was lost on the way and nothing was charged; the payment service crashed or stalled, and nobody knows whether it charged the card; or the card was charged and only the reply was lost.
If the order service simply tries again, it might charge the customer twice. If it gives up, the customer may have paid for an order that was never placed.
The safe ways out are to make the request idempotent, meaning that sending it twice has the same effect as sending it once, or to ask the payment service afterwards what actually happened. Idempotency usually works by sending a unique request number with the charge: the payment service remembers the numbers it has handled and never charges the same one twice.
The lesson: on a network, silence is not an answer. Every remote call needs a time limit and a plan for not knowing, which is why this chapter keeps returning to retries and duplicates.

The eight fallacies of distributed computing

The fallacies of distributed computing are eight assumptions that people new to distributed systems make, all of which are false and all of which cause trouble eventually. The list came from Sun Microsystems: L Peter Deutsch drew up seven in 1994, building on four that colleagues including Bill Joy had already noted, and James Gosling, the creator of Java, added the eighth around 1997. Three decades later every item still describes a real outage somewhere this week.

The eight fallacies, with a consequence and a design response for each

Fallacy	What goes wrong when you believe it	What to do instead
1. The network is reliable	A payment call hangs; the calling thread waits for ever; under load all threads are soon waiting and the order service stops answering anyone.	Set a timeout on every remote call, retry only idempotent operations, put queues where work can wait (resilience patterns).
2. Latency is zero	A basket page that made 40 in-process calls now makes 40 network calls at 2 ms each; 80 ms appears from nowhere, and far more at the tail.	Design coarse-grained interfaces, batch requests, call independent services in parallel, cache stable data.
3. Bandwidth is infinite	Every `MenuUpdated` event carries all the dish photographs; a busy evening saturates the link and delays location updates sharing it.	Send identifiers and the fields consumers need, paginate, compress, keep bulk data next to the code that processes it.
4. The network is secure	Services inside the data centre accept any request without authentication; one compromised container can call the refund API.	Encrypt and authenticate service-to-service traffic (for example mutual TLS) and authorise every call: zero trust.
5. Topology doesn't change	An IP address hard-coded in configuration stops working when the container is rescheduled to another machine.	Use service discovery and DNS names, and expect instances to come and go every minute.
6. There is one administrator	A cloud provider, a partner's firewall team or another department changes a rule, rotates a certificate or lowers a rate limit, and your calls start failing.	Know who controls each hop, monitor the dependencies you do not own, agree contracts and change notice.
7. Transport cost is zero	Serialisation burns CPU, and data crossing availability zones, regions or the public internet is billed per gigabyte; the network bill grows faster than traffic.	Measure the cost of data flows, keep chatty traffic local, choose compact formats where volume is high.
8. The network is homogeneous	A service assumes every client speaks the same protocol version, character encoding and clock; a mobile network, an old partner system or a proxy breaks the assumption.	Use standard, explicitly versioned formats and protocols, and test interoperability with the real clients.

What distribution costs

Every reason to distribute is bought with the same currency. Latency is the first cost. A function call inside a process takes nanoseconds; a network round trip between machines in the same data centre typically takes from a few hundred microseconds to a millisecond or more, before any serialisation or work. Between continents physics sets the floor: light in optical fibre travels at roughly 200,000 km per second, so a London to New York round trip (about 5,600 km each way) cannot take less than about 56 ms. Chapter 2's tail-latency arithmetic applies to every hop.

Observability is the second. In a monolith a stack trace shows the whole story of a failed request. In a distributed system the story is scattered across the logs of many services on many machines. Observability, the ability to understand what the system is doing from its outputs, has to be designed in: correlation identifiers passed on every call, centralised logs, metrics per service and distributed tracing (Chapter 9).

Consistency is the third. A single database transaction can update an order and its payment atomically. When the order and the payment live in different services with different databases, there is no shared transaction; the system must accept a period during which the two disagree and must make sure they converge. Later sections cover the tools: consistency models and sagas and the outbox pattern.

Operational load is the fourth, and the one most often underestimated. Each service needs a build pipeline, a deployment, configuration, secrets, monitoring, alerts, capacity planning, security patches and someone on call. Testing changes too: a change is only safe if every consumer of its interface still works, which calls for contract tests and backward-compatible changes. These costs are roughly proportional to the number of deployable units, which is why the number of services matters.

Doing the arithmetic: synchronous chains

The cheapest moment to discover that a distributed design is too fragile or too slow is on paper. The tool is the series rule from Chapter 2: if a request needs every component in a chain to work, the chain's availability is the product of the components' availabilities, and its latency is roughly the sum of theirs.

A five-service synchronous chain

Five services that each meet 99.9% produce a request path that meets only about 99.5%, and five times the latency of one.

Worked example

Worked example: the availability and latency budget of a call chain

A food-delivery platform handles Place order with a synchronous chain: the API gateway calls the order service, which calls the menu service, which calls pricing, which calls payment. Each service on its own meets 99.9% availability (time-based, 30-day month of 43,200 minutes), answers in a median of 20 ms of its own work, and has a p99 of 250 ms. Each network hop adds about 2 ms. Assume, generously, that failures are independent and that any failure fails the whole request.

One service	Availability 0.999 = 99.9%; allowed downtime 0.001 × 43,200 = 43.2 minutes a month.
Two in series	0.999 × 0.999 = 0.998 (99.8%); about 86 minutes a month.
Five in series	0.999⁵ ≈ 0.99501 (99.50%); unavailable about 0.005 × 43,200 ≈ 216 minutes, or 3.6 hours, a month. Five individually respectable services have produced a two-nines-and-a-half system.
Ten in series	0.999¹⁰ ≈ 0.99004 (99.0%); about 430 minutes, over 7 hours, a month.
Median latency, five in series	Latencies add: roughly 5 × (20 ms + 2 ms) ≈ 110 ms, against 20 ms for the same work done in one process. (Strictly, medians do not add exactly; the sum is a good first estimate.)
Tail latency, five in series	The chance that a request meets at least one hop in its slowest 1% is 1 − 0.99⁵ ≈ 4.9%, so about 1 request in 20 is slowed by at least one 250 ms hop. With ten hops it is 1 − 0.99¹⁰ ≈ 9.6%.
What it takes to keep 99.9% end to end	Each of five services would need about 99.98% (0.9998⁵ ≈ 0.9990): roughly 8.6 minutes of downtime a month each, which in practice means automated recovery.
Design responses	Call menu and pricing in parallel (latency becomes the slower of the two, not the sum); cache menu data in the order service; make payment confirmation asynchronous so the order is accepted and paid shortly after; remove hops that only forward requests.

Chapter 2 introduced the rule that availabilities multiply in series; distribution is where that rule starts to bite, because every network call adds a component to the series.

The independence assumption flatters the result. Services that share a database, a cluster, a configuration system or a release train fail together, and a slow callee can exhaust the caller's threads and spread the outage upwards.

The cure is rarely to make each service heroically reliable. It is to shorten synchronous chains and let parts that need not answer immediately work from a queue.

The architecture quantum, revisited

Chapter 5 introduced the architecture quantum: an independently deployable part with high functional cohesion, including its data, to which one set of architecture characteristics applies (Architectural styles, patterns and tactics). In Software Architecture: The Hard Parts (2021) Ford, Richards, Sadalage and Dehghani sharpen the idea with two kinds of coupling. Static coupling is what must be present for a part to be deployed and run: its code, libraries, database and infrastructure. Dynamic coupling is how parts depend on each other while running: whether they call synchronously or asynchronously, and how they coordinate and stay consistent.

The worked example shows why the dynamic side matters. For the duration of a synchronous call, the caller cannot complete its work unless the callee is up and quick; the two are bound together operationally, and their availability and latency combine as calculated above. Five separately deployed services in a synchronous chain are five deployment units, but for the availability of Place order they behave like one fragile unit. This dependence on the other party being available at the same moment is temporal coupling. Asynchronous messaging removes much of it: if the order service puts a message on a durable queue, the payment service can be down for a minute and catch up afterwards, and the customer still gets an order number. The cost is that the customer does not get a definitive answer immediately, which is a product decision as much as a technical one.

This is the deepest lesson of the section. Drawing service boundaries is half the design; deciding which interactions must be synchronous, and keeping that set small, is the other half.

Watch out

Common mistakes when deciding to distribute

Distributing to fix a code-quality problem. A tangled monolith split into services becomes a tangled distributed system, with network failures added. Fix the boundaries first; Chapter 5's modular monolith is the usual route.
Treating remote calls like local ones. Libraries that make a remote call look like a method call hide latency and failure; Martin Fowler's 'first law of distributed object design' was simply not to distribute your objects.
Counting services instead of synchronous hops. Ten services that mostly exchange events can be more robust than three that call each other in a chain.
Assuming independence of failures. Services sharing a database, a Kubernetes cluster or a configuration store fail together; the series formula is then optimistic and the parallel formula is fiction.
Ignoring the people cost. Ten services need ten sets of pipelines, dashboards and on-call rotas. If one team of six owns all of them, the main benefit of distribution, team autonomy, is absent while every cost remains.

Checklist: before you distribute

Use these questions before splitting a system, or a part of one, across the network.

Name the driverWhich reason in the table above applies, to which part, with what evidence? 'Microservices are modern' is not a driver.
Try the cheaper fix firstCould replicas, caching, a read replica, a background worker or a clearer module boundary inside the monolith meet the need?
Draw the call graphCount the synchronous hops in each key user journey and compute the availability and latency budget, as in the worked example.
Decide who owns which dataEvery piece of data needs one owning service; anything that must change atomically with it probably belongs in the same service.
Design for partial failureTimeouts, retries with idempotency, fallbacks and what the user sees when a dependency is down, decided before the first outage.
Check the prerequisitesAutomated provisioning and deployment, monitoring, centralised logs and distributed tracing must exist, or be planned and funded.
Match teams to boundariesEach service should have one owning team that can change, deploy and support it without asking others.

Check your understanding

A service calls a stock service with no timeout. The stock service becomes very slow rather than failing. Why can this be worse than if it had crashed?

A crashed service fails fast and callers can react; a slow one keeps callers waiting, so their threads and connections fill up until they stop answering too, spreading the outage as a cascading failure.

Three services each have 99.95% availability and are called in series on every request. What is the approximate availability of the request path?

0.9995³ ≈ 0.9985, about 99.85%, assuming independent failures.

Why does the fallacy 'transport cost is zero' matter even when latency and bandwidth are fine?

Moving data costs CPU for serialisation and money for traffic between zones, regions or providers, so a chatty or bulky design can be expensive even when it is fast enough.

Why can five separately deployed services still behave like one unit for availability purposes?

If a request calls them synchronously in a chain, all five must be up at the same moment; their availability combines, so they are temporally coupled despite separate deployment.

Next · 6.2Service-based and SOACoarse-grained domain services sharing a database, the enterprise service bus era of SOA and why it faded, how microservices differ, and when service-based architecture is the pragmatic middle ground.Continue →

Chapter 6 overview

Sources and further reading

Have a question about this section? ArchitectBot, the course’s AI study assistant, answers from the course first.

Ask: “Our checkout calls six services one after another. How do I work out whether that is too fragile, and what should we change first?”