Be Wise Apps
Leer en español · Lire en français · Auf Deutsch lesen
Section 6.8 of 6.8 · Intermediate · 28 min read
Serverless and space-based architectures
Video lesson
5 min · narrated, with captionsListen only (audio)
Read the transcript
This lesson covers two styles that answer the capacity question in opposite ways. Serverless pays only for use. Space-based architecture serves extreme peaks from memory. Both are specialised tools, and both are often chosen for the wrong reasons.
A taxi rank charges only for each journey. When nobody needs a ride you pay nothing, but on a busy night each trip costs more than owning a car would. And the first taxi may take a moment to arrive. That's serverless. Now picture a stadium box office before a huge concert. Instead of every clerk checking one central ledger, each counter keeps its own copy of which seats are free, and sends its sales to the back office later. That's space-based architecture, built for the rush.
Serverless doesn't mean there are no servers. It means your team doesn't provision, patch, size or pay for idle ones. With functions as a service, events trigger small pieces of your own code. Here, a web request, a queue message, an uploaded photo and a nightly schedule, each trigger a function, whose instances the provider creates, scales and destroys. With backend as a service, fully managed services replace server-side parts: a database, email and file storage. Functions are stateless and short-lived. All durable state lives in managed services, so a serverless system is a distributed system by construction.
With no traffic, nothing runs and no compute is billed. As events arrive, many instances start in parallel, and a thousand of them can overwhelm a database with connections. You pay per request, plus memory multiplied by running time. Code waiting on a slow API is paying to wait. The first request to a new instance waits while it starts, from tens of milliseconds to several seconds. These cold starts hit sudden bursts hardest. Functions keep nothing between calls, run under hard limits, and many triggers deliver at least once, so handlers must be idempotent. And serverless ties you closely to your provider. A middle path is to keep business logic in plain modules behind ports, with thin function handlers as adapters.
Is it cheaper? The section works it through with illustrative round-number prices, not any provider's real ones. Two always-on virtual machines and a load balancer cost a flat hundred pounds a month. As functions, the same menu API costs about three pounds twenty per million requests, rising with traffic. The lines cross at about thirty one million requests a month, roughly twelve a second. Below that, serverless is cheaper, because idle time is free. Above it, a steadily busy service pays more. This leaves out people, whose time can cost more than the difference. So redo the sums with real prices and measured traffic.
Space-based architecture takes the database, which every request usually touches, off the hot path. Here, for a ticket on-sale, a messaging grid routes each buyer's request to a processing unit. Each identical unit holds the logic and seat data in memory, and a data grid replicates changes between units. Changes reach the database later: data pumps queue them, and data writers write them. Data readers load data from the database to warm the grid on start-up. The catch is data collisions: two units can update the same item before replication catches up. For data that must never collide, such as the last seat, route all its requests to one unit.
Picture a stadium tour going on sale at ten o'clock, with about four hundred thousand people waiting. The system must never sell a seat twice, and must not fall over. Requests are routed by show and section, so one unit handles all the holds for a section. Holds can never collide. When a buyer chooses two seats, the unit holds them for eight minutes, in memory, in microseconds. Payment runs with an idempotency key. On success the seats are marked sold, and a data pump queues the sale. The database is written at a steady rate it can sustain, and the queue drains a few minutes after the peak. No buyer waits for the database.
Serverless fits spiky, unpredictable or low-volume work, event-driven glue, scheduled jobs, and teams without operations capacity. It fits poorly with steady high-volume traffic, long-running processes, and strict tail latency that cold starts would breach. Space-based architecture is complex and costly, and the database lags behind the grid. Use it only for extreme peaks. First try caching, read replicas, or a virtual waiting room that admits buyers at a controlled rate. That's far simpler, and often enough on its own.
That completes Chapter 6. When you're ready, take the optional chapter test: twenty multiple-choice questions and two written answers, with a pass mark of eighty per cent. Then continue to Chapter 7, on data architecture. See you there.
What you need to know
- Serverless combines functions as a service, where the provider runs event-triggered code and scales it from zero, with backend as a service, where managed services replace server-side components; AWS Lambda (announced 2014) made FaaS mainstream.
- Functions are stateless and short-lived, billed per invocation and per GB-second, subject to cold starts and hard limits on time, memory and payload, and often triggered at least once, so handlers must be idempotent.
- All durable state lives in managed services, so a serverless system is a distributed system by construction and is tightly coupled to its provider unless business logic is kept behind ports.
- Per-request pricing is cheaper than always-on capacity below a crossover volume and more expensive above it; the crossover depends on duration, memory, traffic shape and people costs.
- Space-based architecture serves requests from processing units with replicated in-memory data grids, managed by virtualised middleware, and updates the database asynchronously through data pumps and writers, with data readers to warm the grid.
- Its name comes from the tuple space of Gelernter's Linda (1985); its main risks are data collisions between replicas, memory limits, database lag, cost and complexity.
- Use space-based architecture only for extreme, spiky load that caching, replicas and waiting rooms cannot handle, and route or partition contended data so it cannot collide.
Two answers to the question of capacity
Every style in this chapter so far assumes that someone runs servers and sizes them for the load. This section covers two styles that answer the capacity question in opposite ways. Serverless architecture hands servers to a cloud provider entirely and pays per use, which suits work that is small, event-driven and often idle. Space-based architecture keeps everything a busy request needs in memory, spread over many identical units, so that the database is no longer on the hot path, which suits extreme and sudden peaks such as a ticket on-sale. Both are specialised tools; both are often chosen for the wrong reasons.
Serverless: functions and managed back ends
Serverless does not mean there are no servers; it means the team does not provision, patch, size or pay for idle servers. Mike Roberts' widely cited 2018 article separates two ingredients. Functions as a service (FaaS) runs small pieces of the team's own code in response to events, in containers that the provider creates, scales and destroys, and that may live for only one invocation. AWS Lambda, announced in November 2014 and generally available in April 2015, made the model mainstream; Azure Functions, Google Cloud Run functions and Cloudflare Workers are other examples. Backend as a service (BaaS) replaces server-side components with fully managed third-party services that the client or function calls directly: authentication (Auth0, Amazon Cognito), databases (Firebase, DynamoDB), object storage, email and queues.
A serverless application is usually a combination: functions hold the business logic that is specific to the organisation, and everything else, including all durable state, lives in managed services. The provider handles capacity, availability across zones and the operating system. The team still owns its code, its data, its configuration and its security settings, as the shared responsibility model in servers and the cloud explains.
Events trigger functions; state lives elsewhere
How functions behave: the characteristics that shape design
Event triggers. Functions run in response to events: an HTTP request through a gateway, a message on a queue or stream, a file landing in object storage, a database change, or a schedule. Triggers that invoke the function asynchronously, such as queues and storage notifications, typically deliver at least once and retry on failure, so handlers must be idempotent, exactly as in event-driven architecture.
Scale to zero and scale out. With no traffic, no instances run and nothing is billed for compute. As events arrive, the platform starts more instances in parallel, up to an account or function concurrency limit; on AWS Lambda the default regional limit is 1,000 concurrent executions at the time of writing, raisable on request. Rapid scale-out is a feature and a hazard: a thousand concurrent functions can open a thousand database connections and overwhelm a relational database designed for a few hundred, which is why connection proxies and per-function concurrency caps exist.
Per-invocation billing. Cost is usually a small charge per request plus a charge for memory multiplied by execution time (gigabyte-seconds). Code that waits idly on a slow downstream API is therefore paying to wait.
Cold starts. The first request to a new instance must wait while the platform starts a container, loads the runtime and runs initialisation code: from tens of milliseconds to several seconds, depending on language, package size and network set-up. Instances are reused for later requests while warm, so cold starts hit infrequent traffic and sudden bursts hardest. Mitigations include smaller deployment packages, lighter runtimes, initialising outside the handler, and paying to keep instances pre-initialised (AWS calls this provisioned concurrency), which partly reintroduces always-on cost.
Limits. Functions run under hard limits on execution time, memory, payload size and temporary disk; for AWS Lambda at the time of writing, 15 minutes, 128 MB to 10,240 MB of memory (with CPU allocated in proportion) and 6 MB for a synchronous request or response. Long jobs must be split into steps, and large payloads passed by reference to object storage.
Statelessness. Nothing in an instance's memory or local disk is guaranteed to survive to the next invocation. Caching in memory between calls is an optimisation that may or may not hit, never a place for data that matters.
Where the state goes
Because functions keep nothing, architecture work shifts to choosing where each kind of state lives. Durable business data goes to a managed database, often one that handles many short-lived connections well (key-value or document stores, or a relational database behind a connection proxy). Large objects go to object storage, passed between functions by key. Short-lived shared state, such as rate-limit counters, goes to a managed cache. The progress of a multi-step process goes to a workflow service (AWS Step Functions, Azure Durable Functions or similar), which acts as a durable saga orchestrator, as described in sagas. User sessions go into tokens held by the client or into a session store.
The consequence is that a serverless system is a distributed system by construction, with all the concerns of this chapter: network latency between every function and its data, partial failure, duplicate events, eventual consistency and the need for distributed tracing. A single business action may cross a gateway, three functions, a queue and two databases. Observability (see observability) is not optional.
Vendor coupling: the price of not running servers
Serverless couples a system to its provider more tightly than renting virtual machines or running containers. Event formats, trigger configuration, identity and permission models, workflow definitions and the managed services themselves are all provider-specific. Moving a mature serverless application to another cloud is closer to a rewrite than a migration.
Practitioners disagree about how much this matters. One view holds that lock-in is the price of a large reduction in operational work and that most organisations never move clouds, so optimising for portability wastes effort. The other holds that pricing changes, regulatory demands or acquisitions do force moves, and that deep coupling removes negotiating power. A middle path is to accept coupling at the edges and contain it in the core: keep business logic in plain modules behind ports, with thin function handlers acting as adapters (the hexagonal architecture approach), keep infrastructure as code, and record the coupling as an explicit decision. Cost and lock-in are discussed further in FinOps and lock-in.
Worked example
Worked example: when does always-on become cheaper?
The food-delivery platform's restaurant-menu API could run as functions or on two always-on virtual machines. All prices are illustrative round numbers chosen for the arithmetic, not current list prices of any provider. Each request runs for an average of 200 ms with 0.5 GB of memory, so it uses 0.1 GB-seconds.
| Illustrative serverless prices | Compute £0.00002 per GB-second; £0.20 per million invocations; £1.00 per million requests through the HTTP gateway. |
|---|---|
| Serverless cost per million requests | Compute: 1,000,000 × 0.1 GB-s × £0.00002 = £2.00. Invocations £0.20. Gateway £1.00. Total £3.20 per million. |
| Illustrative always-on cost | Two small VMs at £40 a month each (two for redundancy) plus a load balancer at £20 = £100 a month, sized for peaks of about 300 requests a second. |
| 1 million requests a month (0.4 a second on average) | Serverless £3.20; always-on £100. Serverless is about 30 times cheaper. |
| 10 million a month (about 4 a second) | Serverless £32; always-on £100. |
| Crossover | £100 ÷ £3.20 ≈ 31 million requests a month, about 12 requests a second on average (31.25 million ÷ 2.59 million seconds in 30 days). |
| 100 million a month (about 39 a second) | Serverless £320; always-on still £100, because two VMs still have headroom. |
| Effect of halving duration to 100 ms | Compute falls to £1.00, total £2.20 per million, and the crossover moves to about 45 million. Optimisation pays directly in serverless. |
Below the crossover, serverless is cheaper because idle time is free; above it, a steadily busy service pays more for the same work. Bursty traffic with long idle periods favours serverless even at higher volumes.
The raw comparison leaves out the largest cost for many teams: people. Patching, scaling and on-call for VMs can easily cost more than the difference, which pushes the real crossover higher. Data transfer, logging and observability charges apply to both and are often larger than either compute bill.
Because the result depends on duration, memory, traffic shape and current prices, redo the calculation with real prices and measured traffic before deciding, and again as traffic grows.
The crossover, drawn
When serverless fits, and when it does not
Serverless fits spiky, unpredictable or low-volume workloads, where paying for idle servers is wasteful; event-driven glue, such as reacting to uploads, queue messages and database changes; scheduled jobs and back-office automation; prototypes and new products, where the team should spend its time on the product rather than on infrastructure; and teams without platform or operations capacity.
It fits poorly with steady, high-volume traffic, where per-request pricing overtakes always-on capacity (as in the worked example); long-running or stateful processes, such as video transcoding beyond the time limit or WebSocket servers holding many connections; strict, low tail-latency requirements that cold starts would breach; workloads needing special hardware or fine control of the runtime; and fine-grained designs in which many functions call each other synchronously, which add network hops, cost and failure points at every step. Chapter 1 described how Amazon Prime Video, in 2023, moved one monitoring service from distributed serverless steps to a single process and reported an infrastructure cost cut of over 90% (why architecture matters): not a verdict on serverless, but a reminder that fit matters more than fashion.
Space-based architecture: taking the database off the hot path
Most architectures scale the web and application tiers easily but eventually hit the database, which every request touches. Adding servers in front only moves the queue to the database. Space-based architecture, as described by Mark Richards and Neal Ford in Fundamentals of Software Architecture, removes the database from the synchronous path altogether. Each request is handled entirely in memory by one of many identical processing units, and the database is updated asynchronously, behind the scenes.
The name comes from the tuple space, a shared associative memory introduced by David Gelernter's Linda coordination language in 1985: processes communicate by writing tuples into a shared space and reading or taking matching tuples out, without knowing about each other. Sun's JavaSpaces brought the idea to Java in the late 1990s, and commercial platforms built distributed, replicated spaces on it, which evolved into today's in-memory data grids such as Hazelcast, Apache Ignite and Oracle Coherence.
The parts of a space-based system
- Processing unitA deployable unit containing the application logic and an in-memory data grid holding the data that logic needs, plus a replication engine. Units are identical, so adding one adds capacity. In the ticketing example, each unit holds the seat map and current holds for the events it serves.
- Virtualised middlewareThe infrastructure that manages the units. The messaging grid receives requests and routes each to an available unit. The data grid replicates changes between units' in-memory copies. The optional processing grid coordinates requests that need several kinds of unit. The deployment manager starts and stops units as load rises and falls.
- Data pumpsAsynchronous channels, usually persistent message queues, that carry each change made in a unit to the database side. If the database is slow or down, changes wait in the queue; buyers are unaffected.
- Data writersComponents that consume the pumps' messages and write the changes to the database. They can be scaled and paused independently of the units.
- Data readersComponents that load data from the database into the units when the grid starts cold, for example after every unit has been restarted or redeployed. They are not used in normal request processing.
Space-based architecture for a ticket on-sale
Data collisions and other costs
Holding replicated copies in many units brings the consistency problems of the previous sections into memory. With a replicated cache, each unit has a full copy and changes are copied asynchronously, which gives fast local reads, but two units can update the same item before replication catches up. Richards and Ford call this a data collision and estimate its rate as the number of units multiplied by the square of the update rate, divided by the number of items, multiplied by the replication latency. With 5 units, 20 updates a second, 50,000 items and 100 ms latency, that is 5 × 400 ÷ 50,000 × 0.1 = 0.004 a second, about 115 collisions in an eight-hour day; cutting latency to 1 ms brings it to about one.
For data that must never collide, such as the last seat in a row, the alternatives are a distributed cache, in which each item lives in one partition and every unit reads and writes it there (consistent, but every access is a network call and the grid's partitions must stay available), or routing all requests for one event or section to the same unit, so that only one unit ever changes those seats. Most real designs combine the two: partition the hot, contended data by key and replicate read-mostly data such as venue maps and prices.
Other costs are significant. Data held in memory is limited by memory size and must be rebuilt after a full restart. The database is eventually consistent with the grid, so reporting and other systems see changes late, and losing queued changes is a real risk that the pumps' durability must cover. Testing at realistic peak load is expensive and hard. And the style is complex and costly to build: Richards and Ford rate it low for simplicity and testability and high for cost. It is worth it only when the load genuinely cannot be served by caching, read replicas and a well-tuned database.
Worked example
Worked example: a stadium on-sale
A stadium tour goes on sale at 10:00. About 400,000 people are waiting; for the first ten minutes, demand is hundreds of times the normal rate, then falls away. The ticketing system must not sell a seat twice and must not fall over.
| Before the on-sale | Data readers load the seat maps for each show into the grid. The deployment manager starts enough processing units for the forecast peak. |
|---|---|
| Routing | The messaging grid routes requests by show and section, so all holds for section 112 of Saturday's show are handled by the same unit. Seat holds never collide. |
| A buyer selects two seats | The unit marks them HELD for this buyer for 8 minutes, in memory, in microseconds, and returns. Seat maps for other buyers update through replication. |
| The hold becomes a sale | The payment step runs (an external call, with an idempotency key); on success the unit marks the seats SOLD and a data pump queues the sale. |
| The database | Data writers write sales at a steady rate the database can sustain. During the peak the queue grows; it drains a few minutes after demand falls. No buyer request waits for the database. |
| A hold expires | The unit releases the seats in memory; the pump queues a release record. |
Many ticketing systems put a virtual waiting room in front of the sale as well, admitting buyers at a controlled rate. It is far simpler and often enough on its own; space-based architecture is for when the admitted load itself is still too high for a database-centred design.
Routing by section is the design decision that turns the collision problem from "rare but possible" into "impossible" for the data that matters most.
Characteristics compared (qualitative, for typical designs)
| Characteristic | Serverless (FaaS and BaaS) | Space-based |
|---|---|---|
| Elasticity | Very high, automatic, including scale to zero | Very high for spikes, but units take time to start and load data |
| Performance | Good when warm; cold starts and network hops add latency | Very high: requests served from memory |
| Cost profile | Pay per use; excellent when idle or spiky, expensive when steadily busy | High: many units, large memory, complex middleware, always on |
| Simplicity | Simple per function; the whole system can become hard to follow | Low: replication, collisions, pumps and cold-start loading |
| Consistency | Whatever the managed data stores provide | In-memory replication is eventual unless contended data is partitioned; database lags behind |
| Testability | Units are easy to test; integration needs cloud environments | Hard: realistic peak testing is expensive |
| Operational effort | Low for infrastructure; high for observability and limits | High |
| Portability | Low: provider-specific triggers and services | Medium: depends on the data grid product |
Checklist: before choosing serverless or space-based
Answer these with measured numbers where you can.
- Traffic shapeIs load spiky, idle much of the time or steady? Serverless rewards idle time; space-based rewards extreme peaks; steady load favours ordinary always-on services.
- Cost crossoverHave you calculated per-request cost against always-on cost with real prices, durations and traffic, including gateway, data transfer and people?
- Latency budgetCan the slowest acceptable response absorb a cold start, or the network hop to wherever the state lives?
- LimitsDo any jobs exceed execution-time, memory or payload limits? Can they be split into steps or passed by reference?
- State and connectionsWhere does each kind of state live, and can the data stores cope with sudden bursts of concurrent connections?
- IdempotencyCan every handler safely run twice for the same event?
- ContentionFor space-based designs, which data is contended, and will it be partitioned or routed so it cannot collide?
- Durability of the lagIf the grid or pumps fail, what is lost? Are pumps persistent, and is the database's lag acceptable to reporting and other systems?
- Simpler alternativesWould caching, read replicas, a queue in front of the database or a virtual waiting room meet the need with less complexity?
- Lock-in decisionIs the coupling to the provider or grid product recorded as a conscious decision, with business logic kept behind ports?
Check your understanding
Why must a function triggered by a queue be idempotent?
Because asynchronous triggers typically deliver at least once and retry on failure, so the same event can invoke the function more than once.
Using the illustrative prices in the worked example, roughly how much would 50 million requests a month cost on serverless, and which option is cheaper?
50 × £3.20 = £160 a month, so the £100 always-on option is cheaper on raw infrastructure cost, before counting people and other charges.
In space-based architecture, what do data readers do, and when?
They load data from the database into the processing units' in-memory grids when the grid starts cold, such as after a full restart or redeployment; they are not used for normal requests.
How does routing all requests for one stadium section to one processing unit help?
Only that unit ever changes those seats, so two units cannot update the same seat before replication catches up, eliminating data collisions for that data.
Sources and further reading
Have a question about this section? ArchitectBot, the course’s AI study assistant, answers from the course first.
Ask: “Our ticketing site falls over every time a big show goes on sale. Should we move to serverless, space-based architecture, or something simpler like a waiting room?”