Premium Video

This video is available to premium subscribers only

Unlock Full Access

A system can handle plenty of traffic and still fail its users if one broken component takes the whole thing down. Handling more load and staying online during failures are related problems, but they are not the same problem.

Availability is about the second one: making sure the service is there when users need it. This chapter covers what availability means, how to measure it, and the main approaches to keep a system available even when something fails. We will use an online store as the example throughout.

Suppose our online store runs on one application server connected to one database. Customers can browse products, add items to their carts, and place orders.

Everything works until the application server crashes. The database is still healthy, but customers can no longer use the store.

We need a way to keep serving customers when part of the system stops working. That is where availability comes in.

Availability measures how often users can access and use a system.

Scalability is about handling more load, while availability is about making sure the service is there when users need it. A store that can process ten thousand orders per minute has solved the first problem. If a single server crash takes it offline for an hour, it has not solved the second.

Availability is closely related to reliability, but they are not the same thing. Reliability measures whether the system continues to work correctly over time. For example, an online store might still be available and accepting orders, but if a bug calculates the wrong order total, the system is available but not reliable.

The basic formula to measure availability is simple: uptime divided by total time, where total time is uptime plus downtime.

Availability = Uptime / (Uptime + Downtime)

For example, suppose our store is available for 364 days and down for 1 day over the course of a year. We divide 364 by 365 and multiply by 100, which gives us roughly 99.73% availability.

Availability = 364 / 365 = 99.73%

But that number alone does not tell the full story. We also need to define what part of the system we are measuring and over what period of time.

For example, users might still be able to browse products even if checkout is completely unavailable. A single availability number for the entire store can hide that failure.

Measuring per operation, over a stated window, shows where the real problem is. Checkout is the operation the business cares most about, and it is the one with the worst number.

You will often hear availability described in terms of "nines." Three nines means 99.9% availability, which allows roughly 8 hours and 46 minutes of downtime per year. Four nines, or 99.99%, brings that down to about 53 minutes. Five nines allows just over 5 minutes, while six nines allows only around 32 seconds.

Each additional nine reduces the allowed downtime by roughly a factor of ten. That gives us much less time to detect failures and recover from them. At four nines, a single incident that takes an hour to fix has already used the whole year's budget.

So higher availability targets come at a cost. They usually require more redundancy, spare capacity, automation, monitoring, testing, and faster recovery mechanisms.

Now let's look at the path a request takes through the system. Suppose every request has to pass through a web server, an application server, and a database.

For the request to succeed, all three components need to be available. We call this a series dependency.

If the failures are independent and each component has 99.9% availability, we multiply 0.999 by itself three times. That gives us an overall availability of roughly 99.7% for the complete request path.

Overall = 99.9% × 99.9% × 99.9% = 99.7%

The effect compounds with every component added. Five components at 99.9% each give a combined figure of about 99.5%, which is almost 44 hours of downtime per year, compared with under 9 hours for a single component.

The important idea is that every required component introduces another point of failure. So even when each individual component is highly available, the availability of the entire system can be lower.

Now suppose we have two application servers, and either one can handle the request. In this case, the servers are working in parallel.

If each server has 99.9% availability, then each is down 0.1% of the time. Assuming their failures are independent, the probability that both servers are down at the same time is 0.1% multiplied by 0.1%, which is 0.0001%.

Failure probability = 0.1% × 0.1% = 0.0001%

Availability = 100% - 0.0001% = 99.9999%

That gives the pair a theoretical availability of 99.9999%, or six nines.

But there are a couple of important assumptions here. Traffic must be able to reach the healthy server, and that server must have enough capacity to handle the extra load. And this availability applies only to these two servers, not to the database or the rest of the request path.

Series and parallel are mirror images. In series, the system fails when any one component fails, so combined availability is lower than the weakest part. In parallel, the system fails only when every redundant component fails together, so combined availability is higher than any single part.

That independence assumption is important. Two servers can still fail at the same time if they depend on the same power source, receive the same faulty deployment, or share a network dependency.

The failures that hit both servers at once come from a few recurring sources.

Hardware failures deserve a specific note. At small scale, a server failure feels rare. At large scale, it is routine. A fleet of thousands of servers sees drives and power supplies fail every week, so the design has to assume that any single machine can disappear at any time.

So simply adding more machines is not enough. That is the core idea behind redundancy: provide another way to handle the requests, and design the system so that it can survive the failure of one server or node completely.

One common way to provide redundancy is an active-passive setup. One component handles traffic, while another stays ready as a standby.

For example, in our store's database, the primary database accepts writes and replicates its changes to the standby.

If the primary fails, a monitoring system detects the failure, promotes the standby to become the new primary, and redirects application connections to it. This process is called failover.

Keeping a single active writer makes data changes easier to coordinate. There is one place that accepts writes, so there is no question about which copy holds the latest order.

The trade-off is that failover takes time. Detecting the failure, promoting the standby, and reconnecting the application can cause a brief interruption for users.

Standby Readiness

How quickly we can recover also depends on how ready the standby is.

A cold standby is not running, so during a failure we first need to start it and may also need to restore or load the latest data.

A warm standby is already running and configured, but it still needs some preparation before it can take over, such as catching up on recent changes and being promoted.

A hot standby stays up to date and is ready to serve almost immediately, which makes failover much faster.

The trade-off is cost. The more prepared the standby is, the more infrastructure and resources we need to keep running.

And just having a standby is not enough. We also need to test it under realistic production load. A backup database is not very useful if, during an outage, we discover that it cannot handle the traffic.

For the application servers, we can use an active-active setup instead. Both servers handle requests through a load balancer, so if one fails, the other is already running and can continue serving traffic immediately.

But the surviving server still needs enough spare capacity to handle the extra load. If both servers are already close to their limits, losing one could overload the other.

We also need to make sure either server can handle any request. That means shared state, such as cart and session data, should live outside the application servers in a shared data store.

This is why stateless application servers work well with active-active setups. Any healthy server can pick up the next request, because nothing the request needs is stuck on the server that failed.

Health Checks and the Load Balancer

The load balancer uses health checks to decide which servers should receive traffic. If a server becomes unhealthy, the load balancer stops sending new requests to it.

But detection is not instant. Health checks run every few seconds, and a server is usually marked unhealthy only after several checks fail in a row. Some requests already in progress, or sent during that window, may still fail.

A good health check should verify that the server can actually handle requests, not just that its process is still running.

And finally, the load balancer itself also needs redundancy. Otherwise, it becomes a single point of failure that can make all of the healthy application servers unreachable.

Managed cloud load balancers are built to run redundantly, so this is handled for you on AWS, Google Cloud, and Azure. In a self-managed data center, teams often run two load balancer instances that share a virtual IP address, so the standby can take over the address when the active one fails.

So far, all of our redundant components could still be running in the same building. That means a facility-wide power or network outage could take them all down at once.

To reduce that risk, we can spread the system across multiple availability zones within the same cloud region. For even greater separation, we can deploy across multiple regions.

But wider separation introduces new trade-offs. Replicating data to a distant region adds network latency, especially if we wait for the remote replica before confirming a write. Asynchronous replication avoids that delay, but a regional failure could leave the backup missing some recent changes.

Using multiple cloud providers adds another layer of isolation, but it also increases operational complexity. Each provider has its own tooling, networking, and failure behavior, and the team has to be fluent in both.

The right level of redundancy depends on the failures we need to survive and how much cost and complexity we are willing to take on. For most applications, running across two or more availability zones is the sensible default. Multi-region is worth it when a regional outage would be unacceptable or when users are spread across the globe.

Failures can also happen because the system is overloaded, not because a component broke.

Suppose a big sale causes a sudden spike in orders, and sending confirmation emails starts slowing down checkout. Each checkout request waits for the email provider to respond, and when the provider slows down under the spike, so does every order.

Instead of sending the email during the request, we can save the order first and place an email job in a durable queue. Background workers process those jobs separately, so customers do not have to wait for the email to be sent before seeing their order confirmation.

If a worker fails, unfinished jobs can be retried by another worker. The queue holds the job until a worker confirms it is done, so a crash in the middle of sending does not lose the email.

Now suppose the product recommendation service of our online store becomes slow. Application requests start waiting on it. As more requests pile up, the application servers run out of threads and connections, and the slowdown spreads to other parts of the store, including checkout. This is called a cascading failure.

To limit the impact, we can use timeouts so requests do not wait indefinitely, and a circuit breaker to temporarily stop calling a dependency that is repeatedly failing.

A circuit breaker tracks the calls to a dependency. While calls succeed, it stays closed and lets requests through. Once failures pass a threshold, it opens and rejects calls immediately, without waiting on the slow service. After a cooldown, it lets a few test requests through to see whether the dependency has recovered.

For recommendations, we can add a fallback that shows the product page without recommendations. Customers can still browse and place orders while the recommendation service recovers.

The recommendation service is still down in this picture. What changed is that its failure stays contained. Checkout, which never needed recommendations, keeps working at full speed.

Design for components to fail while keeping a usable path available for customers. That is the idea behind every technique in this chapter.

Measure availability around the operations users actually care about. Use redundancy and safe failover for critical components, queues for work that can happen in the background, and timeouts and circuit breakers to prevent failing dependencies from bringing down the rest of the system.

That is the core of availability.

As an exercise, take a simple application and trace what happens when each component fails. Identify which failures can take down the entire service, and think about how you would recover from them.