Servers, operating systems, virtual machines, containers and cloud services

A server is just a computer that serves

A server is a computer whose job is to wait for requests and answer them: web pages, API calls, database queries, files. The word describes a role, not a special kind of machine. A laptop running a small web application is a server while it answers requests; a rack-mounted machine in a data centre is a server because that is all it ever does. The same word is also used for the program doing the serving (a "web server" such as nginx), so context matters.

Server hardware has the same parts as any computer (see what software is): processors, memory, storage and network cards. It differs in emphasis. Servers usually have many processor cores, large amounts of memory, redundant power supplies and disks so that one failed part does not stop the machine, remote management so that nobody has to be in the room, and no screen or keyboard at all. They are built to run for years without being switched off.

Most servers on the internet run Linux or another Unix-like operating system; Windows Server is common inside organisations that rely on Microsoft products. A server operating system does the same jobs as the one on your laptop (running processes, managing memory and files, talking to the network) but is usually administered through a text command line over a remote connection, and is configured so that it can be rebuilt automatically rather than by hand.

Data centres: where the servers live

Servers are kept in a data centre: a building designed around their needs. Thousands of machines sit in racks, cooled by air or liquid because they turn almost all of their electricity into heat. Power arrives over more than one supply, backed by batteries for the first seconds of an outage and by diesel generators after that. Network connections come from several providers. Doors need badges, and often fingerprints.

An organisation can run its own data centre or rent rack space in someone else's (called colocation), and before about 2006 that was nearly the only choice. Running servers yourself (on-premises) means buying machines months before you need them, guessing how many, paying for them whether they are busy or idle, and employing people to replace failed disks at night. That cost and inflexibility is the problem the cloud set out to solve.

Virtual machines: many computers inside one

A typical server sits mostly idle if it runs one application. A virtual machine (VM) lets one physical machine pretend to be several. Each VM believes it is a complete computer with its own processors, memory, disk and network card, and runs its own operating system, called the guest. A layer of software called the hypervisor shares out the real hardware among the VMs and keeps them apart, so that a crash or attack in one does not reach the others. Some hypervisors run directly on the hardware (VMware ESXi, Microsoft Hyper-V, KVM, which is part of Linux); others run as an application on a normal operating system (VirtualBox).

Virtualisation changed the economics of servers. One machine can be packed with VMs from different teams or customers. A VM is a set of files, so it can be copied, backed up, moved to another machine and created in minutes instead of being ordered and delivered. Note that this is a different use of "virtual machine" from the language VMs such as the Java Virtual Machine mentioned in what software is: those run programs, these imitate whole computers.

Containers: shipping the application with everything it needs

Every developer has heard "it works on my machine". The program depends on a particular language version, particular libraries and particular settings, and the server has slightly different ones. A container solves this by packaging an application together with everything it needs to run (its runtime, libraries, configuration) into a standard unit that runs the same way on a laptop, a test server or the cloud. The name comes from shipping containers: standard boxes that any crane, ship or lorry can handle without knowing what is inside.

The package is a container image: a read-only, versioned file built from a short recipe (with Docker, a Dockerfile). A running copy of an image is a container. Docker, released in 2013, made this practical for everyday developers, and its image format is now an open standard maintained by the Open Container Initiative, so other tools can build and run the same images.

The key difference from a VM is what is shared. A VM imitates hardware and runs a whole guest operating system. A container is an isolated process: it shares the host's operating-system kernel (the core of the operating system, which controls the hardware, memory and processes) with other containers, and the kernel's isolation features give it its own view of files, processes and network. Containers therefore start in seconds or less and take megabytes rather than gigabytes, so dozens can run where a few VMs would fit. The price is weaker isolation, because every container depends on the one shared kernel; that is why cloud providers commonly run customers' containers inside VMs, combining both.

# Start from an image that already has Python installed
FROM python:3.12-slim
WORKDIR /app
# Bake the libraries into the image
COPY requirements.txt .
RUN pip install -r requirements.txt
# Add the application code
COPY . .
EXPOSE 8000
CMD ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Running one container is easy. Running hundreds across dozens of machines, restarting the ones that crash, adding copies when traffic rises and rolling out new versions without downtime is not. That job is done by an orchestrator, of which Kubernetes is the best known; it is covered in containers and orchestration.

The cloud: renting computing by the minute

Cloud computing means renting computing resources over the internet, on demand, from a provider that runs huge data centres: Amazon Web Services (AWS), Microsoft Azure and Google Cloud are the largest, with many others. The US National Institute of Standards and Technology (NIST) published the definition most people cite, Special Publication 800-145, in 2011. It lists five essential characteristics: on-demand self-service (you create resources yourself, through a website or an API, without phoning anyone), broad network access, resource pooling (many customers share the provider's hardware), rapid elasticity (capacity grows and shrinks quickly) and measured service (you pay for what you use).

The analogy is electricity. Factories once ran their own generators; now they plug into the grid and pay per kilowatt-hour. With the cloud, a start-up can rent a hundred servers for an afternoon to test a launch, and stop paying when it hands them back. That is pay-as-you-go pricing. It turns a large up-front purchase into a running cost, and it rewards systems that can switch capacity off when it is not needed. It also creates a new risk: costs that grow quietly, because nothing stops anyone creating resources and forgetting them.

IaaS, PaaS, SaaS and serverless: how much do you manage?

Cloud services differ mainly in how much of the stack the provider manages for you. NIST defined three service models; a fourth, serverless, became popular after AWS launched its Lambda service in 2014. A popular way to remember them is pizza: you can make it at home, buy a ready-made one to bake, have one delivered, or eat out. Each step hands more of the work to someone else, and takes away some of your choices.

"Serverless" does not mean there are no servers; it means you never see or manage them. You upload a function, say which events trigger it (an HTTP request, a file arriving, a message on a queue), and the provider runs as many copies as needed and charges per execution and per millisecond. It is cheap for spiky or low traffic and needs almost no operations work. Its costs show up elsewhere: the first call after a quiet spell can be slow while the provider starts the function (a cold start), there are limits on how long a function may run, and costs can exceed a plain server's for steady heavy traffic. Chapter 6 discusses serverless architectures in depth.

Between these models sit the managed services: databases, message queues, caches, object storage, search engines and many more, which the provider installs, runs, patches, backs up and scales. Using a managed database instead of installing one on a VM moves a large amount of operational work to the provider. The trade-off, discussed in FinOps and lock-in, is that the more provider-specific services a system uses, the harder it is to move.

Regions, availability zones and content delivery networks

Cloud providers group their data centres by geography. A region is a geographic area (such as Ireland, Frankfurt or Virginia) with its own separate set of data centres, isolated from other regions. Each region is divided into several availability zones: physically separate groups of one or more data centres, each with independent power, cooling and networking, typically kilometres apart but close enough for fast connections between them. A fire, flood or power failure should take out at most one zone.

This gives architects a menu of protection. Running copies of an application in two or three zones of one region survives the loss of a data centre at little extra latency; running in two regions survives the loss of a whole region but makes keeping data in step much harder. Regions also matter for law and latency: data protection rules may require personal data to stay in a particular jurisdiction, and users are served faster from a region near them, because no network can beat the speed of light (see how computers talk).

A content delivery network (CDN) pushes this further for content that does not change per user, such as images, style sheets, scripts and video. It keeps copies on servers in hundreds of locations, called edge locations, so that a user in Nairobi fetches the shop's product photos from a nearby city rather than from a data centre in Europe. The page loads faster, and the shop's own servers handle far fewer requests. Many CDNs also absorb attack traffic and can run small pieces of code at the edge.

Scaling up and scaling out

When a system needs to handle more work there are two basic moves. Scaling up (vertical scaling) means a bigger machine: more cores, more memory. It needs no change to the software, which is its great advantage, but there is a largest machine you can buy, it gets disproportionately expensive near the top, and one machine is still one point of failure. Scaling out (horizontal scaling) means more machines sharing the work, usually behind a load balancer that spreads requests across them and stops sending to any that fail. It can grow far further and survive machine failures, but only if the software is designed for it: for example, a server must not keep a user's session only in its own memory, or save uploaded files only on its own disk, if the user's next request might go to a different server.

The cloud makes scaling out attractive because adding a machine is an API call, and autoscaling can add and remove copies automatically as load changes. Databases are usually the hardest part to scale out, which is why Chapter 7 spends a section on replication and partitioning.

Why this matters for architecture

Where software runs is now an architectural decision rather than an afterthought. The choice between VMs, containers, platforms and functions shapes cost, how quickly the team can release, how the system scales and what happens when a data centre fails. Regions and zones turn availability targets into concrete deployment plans; managed services trade operational work for dependence on a provider. Chapter 9 develops scaling strategies, and Chapter 10 covers cloud service and deployment models, containers and infrastructure in professional depth.