Networks, the internet and the web: IP, DNS, HTTP and the client-server model

Networks and packets

Almost no modern software runs alone. A food-delivery app on your phone talks to the company's servers, which talk to restaurants' tablets, a payment provider, a mapping service and a notification service. This section explains how that talking works, from the bottom up.

A network is a set of computers connected so they can exchange data, by cable, Wi-Fi, mobile signal or fibre-optic link. The internet is a network of networks: millions of separately owned networks (home broadband, mobile operators, universities, cloud providers) that have agreed to carry each other's traffic using common rules called protocols.

Data does not travel as one continuous stream. It is cut into packets: chunks typically up to about 1,500 bytes, each labelled with where it came from and where it is going, like a parcel with a sender's and recipient's address. Devices called routers pass packets from network to network, each choosing a good next hop, much as sorting offices pass parcels on towards their destination. Different packets from the same message may take different routes, arrive out of order or occasionally be lost, and the receiving computer puts them back together. Splitting data into packets lets millions of conversations share the same cables fairly, and lets traffic flow around a broken link.

Addresses: IP addresses and ports

For a packet to arrive, the destination needs an address. Every device on the internet is reached through an IP address (IP stands for Internet Protocol). The older version, IPv4, writes addresses as four numbers from 0 to 255, such as 192.0.2.10; it allows about 4.3 billion addresses, which are not enough for every phone, laptop and sensor in the world. The newer IPv6 uses much longer addresses written in hexadecimal (base 16, using the digits 0 to 9 and the letters a to f), such as 2001:db8::1, and has room for practically unlimited devices. Both are in use side by side. (The example addresses in this section come from ranges reserved for documentation, so they belong to no real machine.)

An IP address gets a packet to the right computer, but a computer runs many programs at once. A port number, from 0 to 65535, says which program should receive it. If the IP address is the street address of an office building, the port is the room number. Well-known services have standard ports: 443 for HTTPS, 80 for plain HTTP, 53 for DNS, 5432 for the PostgreSQL database. A server program listens on a port, waiting for connections; when your browser contacts a website it connects to port 443 at the site's IP address.

TCP and UDP: two ways to deliver

On top of IP sit two main transport protocols, which decide how much care is taken over delivery. TCP (Transmission Control Protocol) gives two programs a reliable, ordered stream of bytes. Before any data is sent, the two sides set up a connection with a three-step handshake. Each packet is numbered; the receiver acknowledges what arrived; anything missing is sent again; everything is handed to the program in the right order. TCP also slows down when the network is congested. It is like a recorded-delivery service with tracking: dependable, but with some overhead.

UDP (User Datagram Protocol) simply sends individual packets, called datagrams, with no handshake, no acknowledgements, no resending and no ordering. It is like posting postcards: cheap and quick, but some may not arrive. That is exactly right when a late packet is useless anyway. In a video call, a fragment of sound that arrives half a second late is worse than one that never arrives, so calls, live streams and online games use UDP and cope with small gaps. DNS lookups, which are tiny questions and answers, also traditionally use UDP.

DNS: the internet's address book

People remember names such as shop.example; networks route by IP addresses. The Domain Name System (DNS) translates one into the other. It is a worldwide, distributed directory: no single computer holds every name. Instead, responsibility is split by the dots in the name, read from right to left. The root servers know who is responsible for each top-level domain such as .com or .uk; those top-level domain servers know which authoritative name servers are responsible for each domain registered under them; and the authoritative servers hold the actual records, such as the IP address of www.shop.example. The root zone is served by 13 named server identities, run by 12 independent organisations through well over a thousand server instances around the world.

Your device does not usually walk this chain itself. It asks a recursive resolver, typically run by your internet provider or a public service, which does the legwork and remembers (caches) answers for a period set by the domain owner, called the TTL (time to live). Because answers are cached at many levels, most lookups are answered in a few milliseconds without reaching the root at all. The same caching explains why a change to a DNS record can take minutes or hours to be seen everywhere.

HTTP: requests and responses

Once a program can reach another, the two need a shared language for what they want. On the web that language is HTTP (Hypertext Transfer Protocol). It follows a simple pattern: the client sends a request; the server sends back one response. Each request names a method (what to do), a path (to what), some headers (extra information as name-value pairs) and sometimes a body (data being sent). Each response carries a status code, headers and usually a body: a web page, an image or data in a format such as JSON (a simple text format for structured data, explained in the section on APIs).

The most common methods are GET (fetch something, without changing it), POST (submit something new, such as placing an order), PUT (replace something at a known address, such as updating a delivery address) and DELETE (remove something). GET, PUT and DELETE are defined to be idempotent: sending the same request twice has the same effect as sending it once, which makes them safe to retry after a network failure. POST is not, which is why a careless retry can place an order twice. Headers carry things such as the content type (Content-Type: application/json), the language the user prefers, cookies (small pieces of data a site asks the browser to store and send back with later requests) and credentials, and caching instructions.

HTTP is stateless: each request is understood on its own, and the server is not required to remember earlier ones. Logged-in sessions are built on top, usually by the client sending a cookie or token with every request. Statelessness is one reason the web scales so well, because any of many identical servers can handle any request.

POST /api/orders HTTP/1.1
Host: shop.example
Content-Type: application/json
Authorization: Bearer eyJhbGciOi...

{"product_id": 42, "quantity": 2}

HTTP/1.1 201 Created
Content-Type: application/json
Location: /api/orders/9917

{"order_id": 9917, "status": "received", "total_pence": 3000}

The split between 4xx and 5xx matters in practice. A 4xx says "do not simply retry; fix the request". A 5xx says "the problem is on our side"; retrying later may work, and the operations team should be alerted when 5xx rates rise.

HTTPS and TLS: privacy and identity

Plain HTTP sends everything as readable text, so anyone on the path, such as someone on the same café Wi-Fi, could read or alter it. HTTPS is HTTP sent through a protected channel created by TLS (Transport Layer Security; the current version, TLS 1.3, was published as RFC 8446 in 2018). TLS provides three things: confidentiality (eavesdroppers see only scrambled data), integrity (tampering is detected) and authentication (you are really talking to shop.example, not an impostor).

Authentication rests on certificates. A site's certificate is a digital document saying, in effect, "this public key belongs to shop.example", signed by a certificate authority (CA) that your browser or phone already trusts. It works like a passport: issued by an authority both sides trust, checked against the holder, and valid only until its expiry date. During the TLS handshake the server presents its certificate, the client checks the signature, the name and the dates, and the two sides agree on fresh secret keys for encrypting the rest of the conversation. If any check fails, the browser shows a warning.

TLS protects data in transit between two points. It does not protect data once it is stored, and it does not prove that the site is honest, only that it is who it claims to be. Today HTTPS is the norm for all web traffic, not just payments, and certificates are often issued free and renewed automatically.

The client-server model

Most of the systems in this course follow the client-server model. A client (a browser, a mobile app, another program) starts the conversation by sending a request. A server waits for requests and answers them. The server holds what must be shared and trusted: the product catalogue, everyone's orders, the rules about prices and refunds. The client handles the user's screen and input.

In a restaurant, diners are clients and the kitchen is the server: diners initiate orders; the kitchen holds the ingredients and the expertise and serves many diners at once. One consequence is that the server must never trust the client blindly. A phone app can be modified by its user, so prices, permissions and stock levels must be checked on the server, however carefully the app also checks them.

A server is often also a client: the shop's web server is a client of its database and of the payment provider. Real systems are chains and webs of such relationships, as the sequence diagram below shows.

Latency and bandwidth

Two numbers describe how fast a network feels, and they are often confused. Bandwidth is how much data a connection can carry per second, such as 100 megabits per second: the width of the pipe. Latency is how long a piece of data takes to get from one end to the other, usually measured as a round-trip time (RTT) in milliseconds: the length of the pipe. A motorway analogy helps: adding lanes (bandwidth) lets more cars through per hour but does not make any one car's journey shorter (latency).

Latency has a floor set by physics. Light in optical fibre travels at roughly 200,000 km per second, about two-thirds of its speed in a vacuum. London and Sydney are about 17,000 km apart in a straight line around the globe, so even a perfect fibre along that line would need about 85 ms one way and 170 ms for a round trip. Real cables do not follow the shortest path and routers add delay, so measured round trips are typically around 250 ms. No upgrade can remove that. If loading a page from Sydney needs ten sequential round trips to a server in London, the user waits two and a half seconds, however fast their broadband is.

What happens when you type a web address

All of these pieces come together every time you visit a website. Here is the sequence when you type https://shop.example/products/42 and press Enter, simplified but accurate.

Why this matters for architecture

The moment a system is split across more than one computer, every interaction between its parts becomes a network call, with latency, the chance of failure and the need for security. A function call inside a program takes nanoseconds and never gets lost; a call across a network takes milliseconds, sometimes far more, and sometimes never returns. Many of the hardest problems in architecture follow from that difference, which is why Chapter 6 opens with the fallacies of distributed computing, such as the belief that latency is zero.

The protocols here also reappear as building blocks: HTTP underpins the web APIs in the next part of this chapter and the communication styles of Chapter 6, while TLS and certificates return in secure communication and zero trust. Placing servers close to users is part of multi-region and edge architecture.