Data Architecture

Welcome to Chapter 7, Data Architecture. Code can be rewritten in months, but data lives for decades. Customer accounts, ledgers, orders and medical histories outlast every framework and every service that touches them. Moving that data is slow, risky and expensive. That is why the decisions in this chapter are among the most significant an architect makes: they are hard to reverse, wide in scope, and decisive for availability, performance and modifiability.

This chapter is for developers and architects who already know the distributed styles of Chapter 6, and who now need to make sound, defensible decisions about data. It builds directly on Chapter 6. Splitting code into services is the easy half. As soon as there are several services, someone has to decide who may change each piece of data, how others see it, and what happens when copies of it disagree. It also assumes the database basics from Chapter 0's section on storing data. If tables, queries and transactions are new to you, start there. And it leads on to Chapter 8, which protects this data and the system as a whole.

The chapter has eight sections, and it follows two running examples throughout: a food delivery platform, with its evening rush, courier locations and menus, and an online bank, where losing or double counting a single payment is unacceptable. Section seven point one, on data ownership, sets the most important rule: every important fact should have one system of record and one writer. It explains why a shared database couples services, however cleanly their code is separated. Section seven point two surveys relational, document, key value, graph, time series and other stores. It argues that the relational database remains the right default for systems of record, and that each specialised store must earn its place. Section seven point three goes beneath the familiar ACID acronym to the anomalies that concurrency causes, such as lost updates and write skew, and to what isolation levels really guarantee in widely used databases. Section seven point four explains how databases copy and split data across machines, what replication lag and failover do to correctness, and how to choose a shard key from the numbers.

Here is the first and most important idea in the chapter, from Section seven point one. The picture compares two ways of giving three services their data. On the left, the ordering, kitchen and delivery services all read and write one shared database of orders, menus and couriers. Any service can change any table, so every table is a public interface, and any schema change needs a coordinated release. On the right, each service has its own database. Each schema is private, there are no joins across databases, and data crosses the boundaries only through API calls and events. Now each owner can change its schema alone. That independence has a price. Joins, transactions and referential integrity across services must be replaced by APIs, events, sagas and reporting pipelines, and the section shows how to read data you do not own.

The second half of the chapter deals with data that is derived from the systems of record, and with change over time. Section seven point five covers Command Query Responsibility Segregation, or CQRS, and event sourcing. These separate the model that accepts changes from the models that answer queries, and store a history of events instead of current state, at a real cost in complexity. Section seven point six is about caching: keeping copies close to where they are needed, from the browser to the database, including the stampedes that can take a system down when a popular entry expires. Section seven point seven accepts that every schema, whether database tables, API payloads or events, will change while the system is running. Compatibility rules and the expand and contract pattern allow that change without downtime. Finally, Section seven point eight moves from running the business to understanding it: warehouses, lakes and lakehouses, stream processing, and data mesh, a way of organising analytical data around domains.

The sections form a chain, and this diagram from Section seven point two shows that chain in one food delivery platform. Ordering and Menus each write to their own PostgreSQL database. These are the only two systems of record, and they hold orders, payments and menus. Everything else is a deliberate copy. Courier positions sit in an in-memory key value store and expire after thirty seconds. Change data capture copies menu changes into a search engine, with a few seconds of lag. And a data warehouse receives orders, menus and sampled courier track points for analytics, hours old. Each copy is faster or more convenient for some purpose, each is stale by some amount, and each can be rebuilt.

That chain leads to the one idea to take away from this chapter. For every important piece of data, you should be able to answer three questions. First, who is its only writer? Second, where are its rules enforced? And third, how stale may each copy of it be? Most data problems in real systems, such as orders that change state backwards, reports that disagree with the ledger, double bookings, and customers who cannot see their own changes, are failures to answer one of those questions deliberately. The techniques in this chapter are ways of answering them well, and every one of them is a trade-off between consistency, availability, performance, cost and the effort of running the system.

By the end of the chapter, you will be able to assign ownership of data, including the awkward cases where several services want to write the same data. You will be able to choose a database for each data set, by its shape, its access patterns, its consistency needs and its scale, and justify the cost of each additional store. You will know how to protect a read, modify and write path with locking, version checks or serializable transactions. And you will be able to plan schema changes, with compatibility rules and expand and contract, so that they need no downtime.

Because the sections form a chain, reading them in order works best. But each one also stands on its own, so you can go straight to the section you need. Allow about three and a half hours to read the whole chapter. At the end of each section there are self-check questions. Answer them in your head before you look at the answers; they show you quickly what has stuck. When you are ready, there is an optional chapter test, which takes about forty five minutes. It has twenty multiple choice questions and two written answers. The pass mark is eighty per cent.

That is the shape of the chapter. Start with Section seven point one, on data ownership and database per service, where the rule of one writer for every fact is set out in full. See you there.