Multi-region, disaster recovery and edge computing

Why run in more than one region

Section 0.7 (servers and the cloud) introduced regions (separate geographic areas) and availability zones (independent data-centre groups inside a region), and section 10.1 showed how providers organise them. Running across several availability zones in one region is the normal baseline for any production system: it survives the loss of a data centre at little extra cost, because zones are close enough for synchronous replication. Running across several regions is a much bigger step, and there are only three good reasons to take it.

Latency. Light in optical fibre travels at roughly 200,000 km per second, so a round trip between London and Sydney (about 17,000 km in a straight line) cannot take less than about 170 ms, and real routes take considerably longer. A request that needs several round trips to a distant region feels slow however fast the servers are. Serving users from a nearby region, or from the edge, is the only fix.

Disaster recovery. Whole regions do fail, usually through a shared dependency rather than a physical disaster. On 19 and 20 October 2025, a race condition in the automation that manages DNS records for Amazon DynamoDB left its US-EAST-1 endpoint with an empty DNS record; DynamoDB errors lasted about three hours, but knock-on failures in launching new virtual machines and in load balancer health checks lasted into the following afternoon. Every large provider has had comparable regional incidents. Disaster recovery plans for them.

Data residency. Data residency requirements, from law, regulators or customer contracts, may require certain data to be stored or processed in particular jurisdictions. The GDPR itself restricts transfers of personal data out of the European Economic Area rather than demanding local storage, but sector rules, national laws and enterprise contracts often go further (see privacy by design). Residency often forces multi-region designs in which data is deliberately not replicated everywhere.

Multi-region is expensive: duplicated infrastructure, cross-region data transfer charges, harder data consistency and more complex operations and testing. Many teams that believe they need it actually need good multi-zone design, tested backups and a CDN.

Recovery objectives and the four strategies

Section 2.5 (availability and reliability) defined the two numbers that drive every disaster recovery design. The recovery time objective (RTO) is how long the service may be unavailable; the recovery point objective (RPO) is how much recent data may be lost, expressed as time. Both come from the business, ideally from the cost of an hour of outage and of a lost order, not from engineers' preferences. Lower values cost more, and the cost rises steeply near zero.

The AWS Well-Architected whitepaper Disaster Recovery of Workloads on AWS: Recovery in the Cloud describes four strategies, in order of increasing cost and complexity. The names are widely used beyond AWS and apply equally to Azure, Google Cloud or a pair of private data centres.

The whitepaper itself gives one rough figure covering both RTO and RPO for each strategy; the table separates them because, with continuous replication, the RPO of pilot light and warm standby is set by replication lag rather than by how much of the application is running. Two warnings apply to every column. Replication is not a backup: an accidental deletion, a bad migration or ransomware is replicated faithfully within seconds, so point-in-time backups, ideally immutable and in a separate account, are needed even in an active-active design. And a strategy that has not been rehearsed is a hope, not a plan. Teams that run a regional failover exercise at least a few times a year discover missing permissions, hard-coded endpoints, expired certificates and quotas that are too small in the recovery region, while it is still cheap to find them.

Replicating data across regions

Stateless application servers are easy to run in two places; the data is the hard part, and the trade-offs are those of consistency, CAP and PACELC at continental distances. Synchronous replication across regions means every write waits for at least one long round trip, perhaps 70 to 150 ms or more, which is acceptable for a few critical writes and ruinous for chatty ones. Asynchronous replication keeps writes fast but means the replica lags, and whatever has not arrived when the primary region fails is lost: the replication lag is the effective RPO.

There are four common data designs. Single primary region: all writes go to one region and replicas elsewhere serve reads and recovery; simple, but distant users pay write latency and failover means promoting a replica (with the split-brain risks described in replication and partitioning). Home region per entity: each customer, restaurant or tenant has a home region where its data is written, which keeps writes local and supports residency, at the cost of routing and of rare cross-region operations. Multi-writer with conflict resolution: every region accepts writes and conflicts are merged, for example by last-writer-wins (the default in DynamoDB global tables) or by conflict-free replicated data types; safe only for data where merging is meaningful, such as a basket, and dangerous for money or stock. Globally consistent databases such as Google Spanner or CockroachDB pay consensus latency on writes to give strong consistency across regions; Azure Cosmos DB lets each application choose among five consistency levels, from strong to eventual.

Most real systems mix these per data set, exactly as section 6.6 recommends: payments and order state in a single primary or home region, menus and restaurant details replicated everywhere for reads, and analytics shipped asynchronously.

Failover, DNS and static stability

Traffic is steered between regions by DNS or by a global load balancer. With DNS-based failover, the DNS service health-checks each region and returns the address of a healthy one; its weakness is caching, because resolvers and clients may hold old answers for longer than the record's TTL. Global load balancers (for example AWS Global Accelerator, Azure Front Door, Google Cloud's global external load balancer, Cloudflare) use anycast addresses announced from many locations, so clients keep one address and the provider shifts traffic within seconds.

Deciding when to fail over is harder than the mechanism. Automatic failover reacts fastest but can be triggered by a monitoring fault or a brief network blip, and failing over a database risks two primaries accepting writes. Many organisations therefore automate detection and the failover steps but keep a human decision to start a regional failover, with a runbook and a time limit. Failing back after the incident, including re-synchronising data written in the recovery region, needs a plan too.

The October 2025 incident illustrates a principle that the AWS Builders' Library calls static stability: a system should keep working during a failure without having to make changes at that moment. Systems that already had enough capacity running rode out the problem; systems that needed to launch new instances, or whose recovery depended on control-plane actions in the affected region, could not recover until the region did. For disaster recovery this argues for pre-provisioned capacity (warm standby rather than pilot light, for critical paths), for failover that depends only on the healthy region's data plane, and for DNS, identity and secrets that do not live solely in the region you are escaping.

Cells: limiting the blast radius

Regions protect against regional disasters; most outages, however, are caused by the system's own changes, overloads and poison inputs, which a second region running the same code and configuration will suffer too. Cell-based architecture limits the blast radius of such failures by running many complete, independent copies of the system, called cells, each serving a fixed subset of customers. A thin router maps each request to its cell, usually by customer or tenant identifier. Cells share nothing at run time: each has its own application instances, queues and databases.

If one cell suffers a bad deployment, a hot customer or data corruption, only that cell's customers are affected: with 20 cells, about 5% of them. Deployments roll out cell by cell in waves, so a bad release is caught in the first cell. The design is described in the AWS Well-Architected whitepaper Reducing the Scope of Impact with Cell-Based Architecture (2023) and is used at large scale by AWS's own services and by companies such as Slack. A related technique, shuffle sharding, gives each customer a random combination of a few workers out of many (with 8 workers and 2 per customer there are 28 combinations), so a customer whose requests crash workers takes down only its own combination, and most other customers share at most one worker with it.

Cells cost something: more instances and databases to run, cross-cell operations (reports across all customers, moving a customer between cells) that need separate machinery, and a router and mapping service that must be extremely simple and reliable. They suit large multi-tenant systems with natural partition keys; for a single-tenant internal application they are over-engineering.

The edge: CDNs and edge functions

Between users and regions sits the edge: the hundreds of locations operated by content delivery networks such as Akamai, Cloudflare, Fastly, Amazon CloudFront and Azure Front Door. A CDN caches static and cacheable content (images, scripts, menu pages) near users, following the HTTP caching rules described in caching, and absorbs traffic spikes and many denial-of-service attacks before they reach the origin.

Edge functions go further by running code at those locations: Cloudflare Workers, Fastly Compute, CloudFront Functions and Lambda@Edge, Akamai EdgeWorkers, and the edge functions of platforms such as Vercel and Netlify. Cloudflare Workers runs code in lightweight V8 isolates, the sandboxes JavaScript engines use for browser tabs, rather than in a container per function, so start-up is very fast and many tenants share one process safely. This course itself runs on Cloudflare Workers. Good uses are request routing and A/B assignment, authentication checks and token validation, header and redirect rules, image resizing, simple personalisation, and API proxies that keep secret keys out of the browser.

The limits are architectural. Edge functions have tight CPU-time and memory limits; edge storage is usually eventually consistent (Cloudflare's Workers KV can take some time to propagate a write globally) or offers strong consistency only for a single coordinating object, as Durable Objects do; and the system of record normally stays in a region. A common pattern is therefore read at the edge, write at the origin: serve menus, restaurant pages and session checks from the edge, and send orders and payments to the home region.