CQRS and event sourcing

From command-query separation to CQRS

The idea starts small. In Object-Oriented Software Construction (first edition 1988), Bertrand Meyer proposed command-query separation: every method should either be a command that changes state and returns nothing, or a query that returns an answer and changes nothing. Asking a question should never change the answer. Martin Fowler points out that the rule has sensible exceptions (popping an item off a stack both changes the stack and returns the item), but as a default it makes code easier to reason about, because queries can be called freely, in any order, without side effects.

Around 2010 Greg Young lifted the principle from methods to whole models and called the result Command Query Responsibility Segregation (CQRS). Instead of one object model, and usually one set of tables, serving both updates and reads, the system has a write model that handles commands and enforces business rules, and one or more read models shaped for the questions users ask. The insight is that writing and reading are different jobs. Writing needs strict rules, small transactions and a normalised structure so that a fact is stored once. Reading needs speed, denormalised shapes that match screens and reports, and often far more traffic. A single model that tries to do both well tends to do both badly once the domain becomes rich.

What CQRS actually changes

On the write side, a command such as WithdrawMoney(account 42, £50) arrives at a command handler. The handler loads the relevant aggregate (the cluster of data that must stay consistent inside one transaction, a term from domain-driven design covered in Section 5.7), asks it to apply the business rules, and saves the result. Commands are requests and can be refused: the account may be frozen or the funds insufficient. On the read side, a query such as show me the last 30 transactions with running balances is answered from a read model built for exactly that screen, with no joins, no business logic and no locks on the write tables.

Something must keep the read models up to date. That job belongs to a projection: code that listens to changes on the write side and updates one read model. A projection may maintain a table of balances, a search index of transactions, a monthly statement document and a fraud-scoring feature store, each from the same stream of changes. Read models are therefore derived data: they can always be thrown away and rebuilt from the write side. That property is what makes it safe to add a new read model next year, or to change the shape of an old one.

CQRS comes in degrees, and the lightest degree is often all a system needs. At the lightest, the write and read models are different classes over the same database, with reads served by views or dedicated query code. In the middle, the read models live in separate tables updated in the same transaction as the write. At the heaviest, the read models live in different stores (a search engine, a document database, a cache) and are updated asynchronously from events, which brings scale and flexibility but also eventual consistency. CQRS does not require event sourcing, a message broker or microservices; those are separate decisions that are often, but not necessarily, made together.

Event sourcing: state as a sequence of facts

Most systems store current state. When Asha withdraws £120, the accounts row changes from £500 to £380 and the old value is gone unless someone also writes an audit row. Event sourcing, described by Martin Fowler in 2005, turns this round: the system stores every change as an immutable event in an append-only log, and current state is whatever you get by replaying the events in order. The events are the source of truth; the balance is a calculation.

Events are named in the past tense because they record facts that have already happened and been accepted: AccountOpened, MoneyDeposited, MoneyWithdrawn. A command (WithdrawMoney) is a request that may be refused; an event (MoneyWithdrawn) can no longer be refused, only compensated by a later event. Each aggregate has its own event stream, numbered from version 1, and the store guarantees that events in a stream are kept in order.

Rebuilding state is a left fold: start with nothing, apply the first event, apply the second to the result, and so on. The apply function must be pure and deterministic: it may not call a clock, a random-number generator or another service, because replaying the same events must always give the same state. Any value that came from outside (an exchange rate, a timestamp, a fraud score) is recorded inside the event when it is first decided.

Writing follows a fixed cycle. The command handler loads the stream, folds it into current state, runs the business rule, and appends the new events with the version it expects to write. If another command appended to the same stream in the meantime, the append fails and the handler retries with fresh state. This is the optimistic concurrency of Section 7.3, applied per stream, and it is why two simultaneous withdrawals cannot both spend the same £100.

# A bank account as a stream of events, folded into current state.
# Amounts are in pence so that no floating-point rounding creeps in.
from functools import reduce

events = [
    {"v": 1, "type": "AccountOpened",  "overdraft_limit": 0},
    {"v": 2, "type": "MoneyDeposited", "amount": 50000},
    {"v": 3, "type": "MoneyWithdrawn", "amount": 12000},
    {"v": 4, "type": "MoneyDeposited", "amount": 7550},
    {"v": 5, "type": "MoneyWithdrawn", "amount": 30000},
]

def apply(state, e):                       # pure: old state + event -> new state
    if e["type"] == "AccountOpened":
        return {"balance": 0, "limit": e["overdraft_limit"], "version": e["v"]}
    sign = 1 if e["type"] == "MoneyDeposited" else -1
    return {**state, "balance": state["balance"] + sign * e["amount"], "version": e["v"]}

def decide_withdraw(state, amount):        # command side: check the rule, emit events
    if state["balance"] - amount < -state["limit"]:
        raise ValueError("insufficient funds")
    return [{"v": state["version"] + 1, "type": "MoneyWithdrawn", "amount": amount}]

now = reduce(apply, events, None)          # {'balance': 15550, 'limit': 0, 'version': 5}
as_at_v3 = reduce(apply, events[:3], None) # temporal query: balance 38000

-- An event store can be an ordinary table. The primary key enforces
-- optimistic concurrency: only one writer can append version 6.
CREATE TABLE account_events (
  account_id  uuid        NOT NULL,
  version     integer     NOT NULL,
  event_type  text        NOT NULL,
  data        jsonb       NOT NULL,
  recorded_at timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (account_id, version)
);

-- The command handler read the stream up to version 5, decided, and now appends:
INSERT INTO account_events (account_id, version, event_type, data)
VALUES ('2f9c0d6e-4a51-4c7e-9a43-6d1f2b7e8a10', 6, 'MoneyWithdrawn', '{"amount": 2000}');
-- If a concurrent command already appended version 6, this fails with a
-- unique violation; the handler reloads the stream and decides again.

What event sourcing gives you

A complete audit trail by construction. In a state-based system the audit log is a second write that someone can forget, and it can disagree with the data. In an event-sourced system the log is the data, so it cannot drift. Regulators in banking, insurance and healthcare value this highly.

Temporal queries. What was the balance at 23:59 on 31 March? or Which orders were in status PREPARING when the kitchen display failed? become folds over a prefix of each stream rather than forensic work on backups.

Rebuilds and new read models. Because read models are derived, a team can fix a bug in a projection, delete its output and replay every event to produce a corrected view. It can also build a read model nobody thought of when the events were first recorded, such as a report of how often customers withdraw cash within an hour of being paid, over the full history.

Debugging and what-if analysis. Copying a stream into a test environment and replaying it reproduces the exact state that caused a bug. Replaying the same events through changed rules shows what the new rules would have done.

A natural source of integration events. The events can feed other services, although the events stored in a stream are best treated as a private detail of the owning service. Publishing them directly couples every consumer to the internal model; many teams translate them into a smaller set of public integration events instead, designed as the long-lived published contracts described in Section 6.4.

Snapshots and long streams

Folding five events is instant; folding 400,000 events for a busy merchant account on every command is not. A snapshot stores the folded state of a stream at a known version, so loading becomes read the latest snapshot, then apply only the events after it. Snapshots are a cache, not a source of truth: they can be deleted and recreated at any time, in the background, without stopping the system. Take them every N events or on a schedule, and store the version of the apply code alongside them, because a snapshot produced by old code is wrong once the fold logic changes.

Before reaching for snapshots, check the stream design. Very long streams often mean the aggregate is too large. A bank account that lives for 40 years can be modelled as a series of shorter streams, one per statement period, each opened with a BalanceCarriedForward event that records the closing balance of the previous period. This mirrors how accountants close books, and it keeps every stream short enough to load quickly.

The costs, honestly stated

Complexity and unfamiliarity. Event sourcing changes how every developer on the team thinks about persistence. Simple questions such as which customers live in Leeds? cannot be asked of the event store at all; they need a projection. Tooling for inspecting, migrating and repairing event stores is less mature than for relational tables, and fewer engineers have run such a system in production.

Eventual consistency of read models. When projections run asynchronously, a user can make a withdrawal and then see a balance that does not yet include it. The usual mitigations are to return the new stream version from the command and let the query side wait until its projection has processed that version (a read-your-writes guarantee, as in Section 7.4); to update the user interface optimistically from the command's result; or to update critical read models synchronously in the same transaction as the append. Projections must also be idempotent and record the position they have processed, because events may be delivered more than once after a crash.

Event versioning. Events are stored forever, so their schema can never simply be changed. When MoneyDeposited gains a currency field, years of old events still lack it. The standard techniques, catalogued by Greg Young in Versioning in an Event Sourced System, include tolerant readers that supply defaults for missing fields, upcasting (a function that converts an old event version into the current shape as it is read, so the domain code sees only one version), introducing a new event type alongside the old one, and, as a last resort, copying a whole stream into a new one with transformed events. Section 7.7 covers the general compatibility rules.

Deleting personal data. An immutable log collides with the right to erasure under data protection law. The common technique is crypto-shredding: personal fields in events are encrypted with a key held per customer in a separate key store, and erasure means destroying that key so the ciphertext can no longer be read. The alternative is to keep personal data out of events entirely and store only a reference to a record that can be deleted. Whether destroying a key counts as erasure is a legal judgement rather than a technical one, so agree the approach with the data protection officer; Section 8.7 covers the wider obligations.

Operational load. Rebuilding a projection over a billion events can take hours, so teams need a way to run the new projection beside the old one and switch over when it catches up. Event stores grow without limit, and storage, backup and replay times grow with them.

When not to use either, and common mistakes

Neither pattern suits a domain that is mostly forms over data, a small team building its first version, or a system whose only problem is read load (a read replica or a cache is far cheaper). Event sourcing is also a poor fit where data must be truly deleted on demand at high volume, or where most queries cut across many aggregates, since every such query needs its own projection.

The common mistakes are predictable. Teams apply the patterns system-wide instead of to one context. They expose stored events as the public contract, freezing their internal model. They put derived values or external calls in the apply function, so replay gives different answers. They design events as generic AccountUpdated blobs, which record that something changed but not what happened or why, and lose most of the value. And they forget that a read model is a cache: they start writing to it directly, after which it can no longer be rebuilt.

Used selectively, the combination is powerful. A food-delivery platform might event-source the order lifecycle (placed, accepted, preparing, picked up, delivered, refunded), because disputes and refunds need the exact history, while keeping menus, restaurant profiles and user settings as ordinary tables. A bank might event-source the ledger and payments while storing customer contact details conventionally, which also keeps most personal data out of the immutable log.