Synchronous and asynchronous communication: REST, gRPC, GraphQL and messaging

Two ways to talk, and the coupling each creates

Once a system is split into separately deployed parts (see why distribute), those parts must talk over a network, and the way they talk shapes the system as much as the way they are divided. There are two basic modes. In synchronous communication the caller sends a request and waits for the reply before it continues, like a telephone call. In asynchronous communication the sender hands a message to an intermediary, usually a message broker, and carries on; the receiver picks the message up when it is ready, like a letter in a post box. Section 0.6 introduced the difference; here it is examined as an architectural decision.

Synchronous calls create temporal coupling: both parties must be running and reachable at the same moment. The caller is therefore at best as available as the product of its dependencies' availabilities, and its response time, including its tail latency, contains theirs: the arithmetic of Section 6.1. A slow payment service makes the order service slow, and if all the order service's threads are waiting on it, the order service stops answering anybody. Timeouts, retries and circuit breakers limit the damage and are covered in resilience patterns.

Asynchronous messaging removes temporal coupling: the broker stores the message, so the receiver can be slow, busy or briefly down without the sender noticing. It does not remove every kind of coupling. Both sides still depend on the message's format and meaning (format and semantic coupling), and on the broker itself, which becomes critical infrastructure. The sender also loses immediate knowledge of the outcome, so the design must cope with work that fails later, messages that arrive twice and messages that arrive in a different order. Asynchrony moves complexity rather than deleting it.

A useful rule of thumb: use a synchronous call when the caller genuinely cannot continue without the answer ("is this card authorised?"), and asynchronous messaging when the caller only needs to know that the work will happen ("send the receipt", "tell the restaurant"). Many good designs keep the user-facing path short and synchronous and push everything else behind a queue.

REST as Fielding defined it

REST (Representational State Transfer) was defined by Roy Fielding in his 2000 doctoral dissertation as an architectural style for large, networked hypermedia systems: the web itself. Fielding derived it as a set of constraints, each of which buys a quality. Client-server separates user interface from data storage so both evolve independently. Stateless interaction means each request carries everything needed to understand it, so any server instance can handle it, which helps scalability and recovery. Cacheable responses say whether they can be reused, cutting latency and load. A uniform interface (resources identified by URIs, manipulated through representations, self-descriptive messages, and hypermedia as the engine of application state) lets generic intermediaries such as caches and proxies understand traffic. A layered system lets proxies, gateways and load balancers sit in the path invisibly. Code on demand, such as JavaScript sent to a browser, is the only optional constraint.

Most APIs described as RESTful satisfy only some of these. Leonard Richardson's maturity model, presented in 2008 and popularised by Martin Fowler in 2010, describes the steps. At level 0, HTTP is merely a tunnel: every call is a POST to one endpoint with the operation named in the body, as in SOAP. At level 1, the API has separate resources (/orders/1042, /restaurants/7). At level 2, it uses HTTP methods and status codes as HTTP defines them. At level 3, responses carry hypermedia controls: links telling the client what it can do next (a placed order contains a link to cancel it, until cancellation is no longer allowed).

Practitioners genuinely disagree about level 3. Fielding has insisted that an API without hypermedia is not REST. Most public APIs stop at level 2 because their clients are written against documentation rather than discovered at run time, and the benefit of following links rarely pays for the extra design. A pragmatic reading is that level 2, done consistently, gives most of the value; hypermedia earns its keep when the server must steer many independently written clients through a workflow that changes.

HTTP semantics that matter for architecture

HTTP's semantics are defined in RFC 9110 (2022). Two method properties matter more than any other for distributed design. A safe method only reads: GET, HEAD and OPTIONS change nothing the client asked to change. An idempotent method may change state, but repeating it has the same intended effect as sending it once: PUT (replace this resource with this representation) and DELETE are idempotent, as are all safe methods. POST is neither, and PATCH is not idempotent in general. The property matters because networks fail between a request being processed and the response arriving. RFC 9110 says that idempotent requests can be repeated automatically after such a failure, and that clients should not automatically retry non-idempotent requests unless they know the request is in fact safe to repeat.

Status codes are part of the contract, because client code branches on them; Section 0.6 introduced the common ones. Success codes should say what happened: 201 Created (with a Location header) when a resource is created, 202 Accepted when work has only been queued, 204 No Content for success with no body. For conflicts, 409 means the request clashes with the resource's current state, 412 that a precondition failed and 422 that a well-formed request is semantically invalid. The architectural rule is about retries: a 4xx response will fail again if the request is sent unchanged (429 Too Many Requests is the exception: wait as told, then retry), while a timeout, 502, 503 or 504 may succeed later. Clients should retry only those, with backoff and jitter, and only when the request is idempotent or carries an idempotency key.

Two further HTTP features solve distributed problems cheaply. Conditional requests give optimistic concurrency control: a client reads an order with its version tag (ETag), then sends If-Match with that tag when updating; if someone else changed the order in between, the server answers 412 and nothing is overwritten. Caching headers (Cache-Control, ETag) let browsers, CDNs and proxies reuse responses, which is often the cheapest performance tactic available.

gRPC: contracts first, binary on the wire

gRPC is a remote procedure call framework that Google open-sourced in 2015 as the successor to Stubby, the RPC system it had used internally for over a decade; version 1.0 followed in 2016. The contract is written in a .proto file using Protocol Buffers, a compact binary serialisation format, and code generators produce typed client stubs and server skeletons in many languages. Calls travel over HTTP/2, which multiplexes many concurrent requests over one connection.

gRPC offers four kinds of call: unary (one request, one response), server streaming (one request, a stream of responses, such as live rider positions), client streaming (a stream of requests, one response, such as uploading telemetry) and bidirectional streaming. Clients set a deadline; when it passes, the call ends with DEADLINE_EXCEEDED, and deadlines can be propagated down a call chain so that work stops everywhere once nobody is waiting for the answer.

Its strengths are efficiency, strong typing and generated code, which make it popular for internal service-to-service calls in polyglot organisations. Its costs are the mirror image. Binary payloads are hard to read in logs and with ordinary tools. Browsers cannot call gRPC directly, so public and web-facing APIs usually need a proxy or a REST or GraphQL facade. HTTP caching does not apply in the usual way. Compatibility depends on discipline with field numbers: a field's number, not its name, identifies it on the wire, so numbers must never be reused, and removed fields should be marked as reserved.

syntax = "proto3";

service Dispatch {
  rpc AssignCourier (AssignRequest) returns (Assignment);
  rpc TrackCourier (TrackRequest) returns (stream Position);
}

message AssignRequest {
  string order_id = 1;
  string restaurant_id = 2;
  reserved 3;            // field removed in v2: never reuse 3
}

message Position {
  double lat = 1;
  double lng = 2;
  int64 recorded_at_ms = 3;
}

GraphQL: the client chooses the shape

GraphQL is a query language and runtime for APIs that Facebook developed internally from 2012 for its mobile apps and released publicly, with a draft specification and reference implementation, in 2015; it moved to the vendor-neutral GraphQL Foundation in 2018. The server publishes a typed schema; clients send queries to a single endpoint naming exactly the fields they want, possibly across related objects, and receive a JSON response of the same shape. Mutations change data, and subscriptions push updates.

Its main benefit is for diverse clients. A food-delivery mobile app on a slow connection can ask for an order's status, the restaurant's name and the courier's first name in one round trip, without receiving the twenty other fields the web dashboard needs. That avoids both over-fetching and the chain of requests a resource-by-resource REST API can require, and it can replace a separate backend for each frontend. The schema is also self-documenting and lets tools validate queries before they run.

The trade-offs are real and are mostly paid by the server team. Each field is filled by a resolver, so a naive implementation that fetches each order's restaurant separately produces the N+1 query problem: one query for fifty orders, then fifty more for their restaurants. The usual fix is batching and per-request caching, as popularised by the DataLoader library. Because clients can compose arbitrarily deep queries, servers need query depth and cost limits, timeouts and often an allow-list of approved (persisted) queries to stop accidental or malicious overload. HTTP caching is weaker when everything is a POST to one URL. Authorisation must be enforced per field or per object rather than per URL. And errors are typically reported in an errors array inside a successful HTTP response, which monitoring based on status codes will miss.

Messaging: queues and topics

Messaging moves the conversation into a broker. As event-driven architecture explains, a queue delivers each message to one of several competing consumers, the standard way to scale a worker pool, while a topic delivers a copy to every subscriber, so the restaurant, dispatch and notification services can each react to OrderPlaced without the order service knowing they exist. That section also covers delivery guarantees, ordering, duplicates and dead-letter queues; queues and back pressure covers how queues absorb load spikes.

Messaging can also carry request and reply: the requester sends a command with a correlation ID and a reply-to address, and matches the eventual reply to its request. This keeps the temporal decoupling of a broker while still getting an answer, at the cost of more moving parts.

Designing APIs that last: resources, pagination and filtering

Model resources around the domain, not the database. Expose the nouns your consumers think in (/orders, /orders/1042/items, /restaurants/7/menu), not tables or internal services. Actions that do not fit create, read, update and delete can usually be modelled as resources themselves: POST /orders/1042/cancellation creates a cancellation, which can carry a reason and be refused with 409 if the food is already on its way. This keeps the uniform interface while expressing the business process honestly.

Paginate every collection, because collections grow. Offset pagination (?offset=40&limit=20) is simple and allows jumping to page 9, but it slows down on deep pages, because the database must skip all earlier rows, and it shows duplicates or skips items when rows are inserted while someone is paging. Cursor pagination returns an opaque token pointing just past the last item (?after=eyJpZCI6MTA0Mn0&limit=20); it stays fast and stable under inserts, at the cost of no random access. For feeds and large or changing data, cursors are the safer default. Always cap the page size on the server.

Filtering and sorting should use predictable query parameters (?status=delivered&placed_after=2026-10-01&sort=-placed_at) and be backed by indexes; an API that lets clients filter on any field invites queries the database cannot answer quickly. Support only the filters consumers need, and document them in the contract.

Idempotency keys: making POST safe to retry

Partial failure (Section 6.1) leaves a client that got no answer unsure whether its request was processed. Retrying a GET is harmless; retrying POST /orders may create a second order and charge the customer twice. An idempotency key turns the unsafe retry into a safe one. The client generates a unique value (typically a random UUID) for each logical operation and sends it in a header such as Idempotency-Key. The server records the key with the outcome of the first request; any later request with the same key returns the recorded outcome instead of doing the work again.

Payment providers popularised the technique, and the IETF HTTP APIs working group has drafted a standard Idempotency-Key header. At the time of writing it has not been published as an RFC (the latest draft, from October 2025, has expired), but its rules reflect common practice: a key reused with a different request body should be rejected with 422, a retry that arrives while the first request is still running should get 409 Conflict so that two copies never run at once, and keys are scoped to a client and expire after a published period, often 24 hours. The worked example applies each rule.

def create_order(request, db):
    key = request.headers.get("Idempotency-Key")
    if not key:
        return problem(400, "missing-idempotency-key")
    body_hash = sha256(request.body)
    with db.transaction():
        row = db.find_idempotency(request.client_id, key)
        if row and row.body_hash != body_hash:
            return problem(422, "idempotency-key-reused")
        if row and row.status == "in_progress":
            return problem(409, "request-in-progress")
        if row:
            return row.saved_response          # replay, do nothing new
        db.insert_idempotency(request.client_id, key, body_hash, "in_progress")
        order = db.insert_order(request.json)
        response = created(order)
        db.complete_idempotency(request.client_id, key, response)
    return response

Errors that machines can read: RFC 9457

Clients need errors they can act on programmatically, and humans need errors they can understand. Rather than every team inventing an error shape, use Problem Details for HTTP APIs, RFC 9457 (July 2023), which replaced RFC 7807 (2016). A problem details object has the media type application/problem+json and five standard members: type (a URI identifying the kind of problem, which can link to documentation), title (a short, human-readable summary of that kind), status (the HTTP status code), detail (an explanation of this occurrence) and instance (a URI identifying this occurrence). Extension members can carry machine-readable specifics such as which fields failed validation.

Two rules keep error responses safe. Client code should branch on type and status, never on the wording of title or detail, which may be translated or improved. And errors must not leak internals: stack traces, SQL fragments and internal host names help attackers more than callers. Log the details with a correlation ID and return the ID instead.

{
  "type": "https://api.fooddelivery.example/problems/outside-delivery-area",
  "title": "Address is outside the delivery area",
  "status": 422,
  "detail": "Restaurant 7 delivers within 4 km; the address is 6.2 km away.",
  "instance": "/v1/orders/attempts/9b1e",
  "max_distance_km": 4,
  "actual_distance_km": 6.2
}

Evolving an API without breaking its consumers

An API with consumers is a promise, and Hyrum's law (Section 0.6) warns that someone depends on every observable behaviour. Change is therefore governed by one principle: make backward-compatible changes freely and avoid breaking ones. Consumers help by being tolerant readers (a term Martin Fowler uses): they ignore fields they do not understand and read only what they need, so additions never break them. The table below summarises the usual rules; the same logic, applied to stored data and event schemas, appears in schema evolution.

When a breaking change is unavoidable, version. The common strategies are a version in the URL path (/v2/orders: explicit, easy to route and cache, but encourages large, infrequent jumps), a custom header or media type parameter (Accept: application/vnd.fooddelivery.v2+json: keeps URLs stable but is less visible and harder to test in a browser), and dated versions pinned per client account, which some large API providers use to release small breaking changes continuously while each client upgrades when it chooses. Internal APIs between services owned by the same organisation often avoid versions altogether through the expand and contract approach: add the new field or endpoint, migrate every consumer, then remove the old one. Whatever the strategy, announce deprecations in advance; the Deprecation (RFC 9745, 2025) and Sunset (RFC 8594, 2019) response headers let a server tell clients, in machine-readable form, that an endpoint is deprecated and when it will stop working.

Describing contracts: OpenAPI and AsyncAPI

A contract written in a machine-readable format can drive documentation, client generation, server stubs, mock servers, linting and tests. OpenAPI is the standard for HTTP APIs. It began as the Swagger specification, which SmartBear donated to the newly formed OpenAPI Initiative under the Linux Foundation in 2015 and which was renamed in 2016; version 3.1 (2021) aligned its schemas with JSON Schema, and version 3.2 followed in September 2025. AsyncAPI applies the same idea to message-driven APIs: it describes channels (queues and topics), the operations that send or receive on them and the message schemas; version 3.0 was released in December 2023.

Teams choose between design-first (write and review the contract, then implement it) and code-first (generate the contract from annotated code). Design-first makes the API a deliberate decision that consumers can review before anything is built, and suits public and cross-team APIs; code-first keeps contract and implementation in step with less effort, and suits internal APIs that change quickly. In either case, keep the contract in version control, lint it in the pipeline and run a breaking-change check that compares each proposed version with the published one.

Contract testing and consumer-driven contracts

End-to-end tests that deploy every service together are slow, flaky and get slower as services multiply, yet unit tests cannot detect that a provider has changed a field a consumer relies on. Contract testing fills the gap: each side is tested in isolation against a shared description of their interactions. In consumer-driven contracts, a pattern Ian Robinson described in 2006, each consumer states the parts of the provider's API it actually uses, and the provider runs all its consumers' expectations as part of its own build. The provider then knows exactly which fields nobody reads (safe to change) and which changes would break a named consumer.

Pact, created at realestate.com.au in 2013, is the best-known tool. The consumer's tests run against a Pact mock provider and record the requests made and the responses expected into a contract file; the provider's pipeline replays those requests against the real provider and checks the responses. A broker stores contracts and verification results, so before deploying, a pipeline can ask whether this version of a service is compatible with the versions of its consumers and providers already running in production. Contract tests do not replace a few end-to-end smoke tests, and they check structure rather than business meaning, but they let services be deployed independently with confidence, which is the point of splitting them up.