Circuit Breaker Pattern
Practice this topic in a realistic system design interview
Remote calls fail in ways local function calls do not. Another service can return errors, time out, or become so slow that callers run out of threads and connections while waiting. The slowdown then spreads to services that were otherwise healthy.
The Circuit Breaker pattern stops a service from repeatedly calling a dependency that is failing. When too many calls fail or become too slow, the breaker trips and new calls fail fast. Later, it lets a few test calls through to see whether the dependency has recovered.
This chapter covers what the Circuit Breaker pattern is, how it protects a system when one of its dependencies starts failing, how its three states work, and the trade-offs involved in configuring it.
Loading simulation...
1. How One Slow Service Can Take Down Many
Suppose we have an e-commerce system built with microservices. When a user places an order, the order service calls the payment service. Normally, the payment service responds in about 100 milliseconds.
Now imagine the payment service becomes degraded. Maybe its database is overloaded. Instead of failing quickly, requests start taking 30 seconds.
- The order service keeps sending calls, and each one waits for a response.
- Every waiting request holds a thread and a connection.
- Before long, all of the order service's threads are stuck waiting for payments.
- Now the order service cannot handle other requests either, even ones that have nothing to do with payments.
- Services that depend on the order service start waiting on it in the same way.
A slowdown in one dependency has now spread through the system. This is called a cascading failure.
2. Why Timeouts and Retries Are Not Enough
The first protection we usually add is a timeout. If the payment service does not respond within two seconds, the order service stops waiting and fails the request.
That helps, but every request still spends two seconds waiting before it fails. Under heavy traffic, that can still tie up a large number of threads and connections. At 100 order requests per second, a two-second timeout means about 200 requests are always sitting and waiting on a payment service that is not going to answer.
Retries can make the problem even worse. Suppose every failed request is retried three times. The payment service, which is already overloaded, may now receive up to four times as many requests. That extra traffic makes recovery even harder.
What we need is a way for the order service to recognize that the payment service is unhealthy and temporarily stop calling it. That is what a circuit breaker does.