In a monolith, one function calls another inside the same process. In microservices, that call crosses the network, so the caller must know where the target service is.

Hardcoded addresses break quickly because instances move, restart, scale up, scale down, and get new IPs.

Service discovery keeps this mapping updated as the system changes.

This chapter covers service registries, client-side and server-side discovery, DNS, Kubernetes service discovery, Consul, Eureka, and registry failure handling.

Consider a single request. An Order service has accepted a checkout and needs to charge the customer, so it has to call the Payment service. Both run as multiple instances for capacity and redundancy.

The Order instance handling this request needs to pick one healthy Payment instance and send it an HTTP request, which means it needs that instance's current network address.

Four forces keep that address from ever being stable. Auto-scaling adds and removes instances continuously in response to traffic, so the set of valid Payment addresses is a moving target by design. Rolling deployments replace every old instance with a new one on each release, retiring addresses and minting new ones several times a day.

Failures take instances down without warning, and the address of a crashed instance must stop receiving traffic immediately or every request routed to it will hang and fail. And in containerized environments, an instance almost never keeps its address across a restart, so even "the same" service comes back at a different place.

The need underneath all of this is a level of indirection. The caller wants to ask for "a healthy Payment instance" by logical name and get back a usable address, without knowing or caring how many instances exist or where the platform happened to place them this minute.

Something has to maintain the live mapping from a stable logical name to the current set of physical addresses, and keep it accurate as instances come and go. That something is the service registry.

The service registry is a database whose entire job is to answer one question: for a given service name, which instances are currently alive and at what addresses? It is the source of truth that everything else in discovery builds on.

Each entry maps a logical service name to an instance's host, port, and usually some metadata like version, zone, or health status. When a caller wants to reach a service, the answer always comes, directly or indirectly, from the registry.

For that answer to be trustworthy, the registry has to track three events in an instance's life.

When an instance starts up, it registers, announcing "I am an instance of Payment, I am at 10.0.4.17:8080, and I am ready for traffic." When it shuts down gracefully, it deregisters, removing itself so no one is sent to an address that is about to go dark.

And because graceful shutdown is not guaranteed, an instance that crashes never gets to deregister, every instance must continually prove it is still alive by sending a periodic heartbeat.

If the registry stops hearing heartbeats from an instance for some timeout, it assumes the instance is dead and evicts it. This heartbeat-and-eviction loop is what keeps the registry from confidently handing out the address of a server that died three minutes ago.

The registration itself is a small payload. A Payment instance announcing itself to a registry like Consul sends its logical name, its host and port, a health-check URL the registry can poll, and a TTL that says how stale a heartbeat is allowed to get before eviction.

The check block is what lets the registry decide on its own that an instance is dead. Every 10 seconds the registry calls GET /health on the instance, and if that endpoint stops returning success for 30 seconds straight, the instance is evicted from the directory. The meta fields ride along so callers can later filter by version or zone.

With self-registration, the instance posts that payload on startup and then renews its lease on a timer. Against Consul's HTTP API the calls are plain:

The register call runs once when the process is ready for traffic, and the deregister call runs in a shutdown hook so a clean exit takes the instance out of the directory at once rather than waiting for the health check to notice it is gone.

There are two ways an instance can get itself into the registry.

With self-registration, the instance registers and heartbeats itself, the application code or a bundled client library talks to the registry directly. It is simple and gives the instance control, but it couples your application to the registry's API.

With third-party registration, a separate component, often called a registrar, watches the deployment platform and registers instances on their behalf, so the instances themselves stay ignorant of the registry. Kubernetes uses this model: you never write registration code, the platform notices your pod and records it for you.

With the registry in place, the open question is who talks to it when a call needs to be made, and that is where client-side and server-side discovery split.

In client-side discovery, the calling service does the work itself. When the Order service needs to call Payment, its own discovery client queries the registry, gets back the full list of healthy Payment instances, applies a load-balancing rule to pick one, and opens a connection straight to that instance. The registry hands over the list; the client chooses and connects.

Concretely, the lookup returns the set of healthy instances and the client picks one. Querying Consul for healthy payment instances returns an address list the client can load-balance over:

Note that the crashed instance is simply absent from this list because its health check stopped passing, so the client never has to test liveness itself. A small Java client turns that list into a call:

The getInstances call returns only the passing instances, loadBalancer.choose applies the routing rule, and the request goes straight to that address. The clear advantage is efficiency. Once the client knows the addresses, it talks to the target instance directly, so the request takes a single network hop with no intermediary sitting in the path adding latency or another thing to fail.

The client also gets full control over load balancing, it can route by latency, prefer instances in its own availability zone to save on cross-zone traffic, or apply weighting during a canary release, all using information only the caller has.

Netflix's original stack is the canonical example here, with Eureka as the registry and Ribbon as the client-side load balancer baked into each service.

The cost is that every client now carries discovery logic. The querying, caching, load balancing, and instance-health handling all live inside the caller, which means that logic has to exist in every language your services are written in.

A polyglot shop running Java, Go, and Python services needs a competent discovery client for each, and they must agree on behavior. This language coupling is the main strike against client-side discovery, and it is the problem the server-side model exists to solve.

In server-side discovery, the caller is deliberately kept dumb. The Order service sends its request to a single, stable address, a load balancer or router, and that intermediary takes responsibility for finding a healthy Payment instance.

The load balancer queries the registry, applies the load-balancing rule, and forwards the request to a chosen instance. The caller never sees the registry and never sees the instance list; from its point of view it just called "Payment" at a fixed address and got a response.

This flips the trade-off. The client becomes trivial, it makes an ordinary request to a known address with no discovery code at all, which means the discovery logic is written once in the load balancer rather than once per language.

A polyglot fleet stops being a problem, since a Python service and a Java service reach Payment exactly the same way.

The price is an extra network hop on every call, because the request now travels through the load balancer instead of going direct, and that intermediary is one more component to run, scale, and keep available.

AWS handles this model with its Elastic Load Balancers, and Kubernetes implements it natively, so teams running on those platforms are doing server-side discovery whether they have named it or not.

The two models answer the same question with opposite instincts about where intelligence should live. Client-side pushes it into the caller for efficiency and control; server-side centralizes it behind an intermediary for simplicity and language independence. Laying them side by side makes the choice concrete.

There is no universal winner. Client-side discovery is the better fit when you control the stack end to end, run a small number of languages, and want the lowest possible latency and the smartest routing.

Server-side discovery is preferable when you have many languages, want to keep service code as simple as possible, or are on a platform that already provides it.

In practice the industry has moved toward server-side, because the service mesh and Kubernetes made it the default, and letting the platform handle it is easier than reimplementing a discovery client in every language. The recommendation depends on the specific context: the language count, the platform already in use, and how much routing control the system needs.

There is an older, simpler approach that predates dedicated registries: use DNS as the directory. Give each service a DNS name like payment.internal, and resolve that name to the addresses of its instances.

It is appealing because DNS is universal, every language and runtime already knows how to resolve a hostname, so there is no special client library to write and no new system to learn. SRV records can even carry port numbers alongside addresses, making DNS a more complete answer than plain A records.

The trouble with DNS is liveness, and it traces back to caching.

DNS was built for records that change rarely, so caching is aggressive and baked in at every layer: the resolver, the operating system, and often the application runtime all hold onto answers for the duration of a record's TTL. That is a feature for a website's address and a liability for a fleet of instances that churns by the minute.

When an instance dies, its address can linger in caches until the TTL expires, and callers keep trying to reach a dead server in the meantime. Cutting the TTL low enough to react quickly, a few seconds, partly fixes staleness but overwhelms the DNS infrastructure with constant re-resolution and still cannot match the speed of a heartbeat-driven registry.

DNS-based discovery is a reasonable, low-effort choice for systems that change slowly, but its caching model makes it a poor fit when instances come and go rapidly and you need fast, accurate liveness.

For containerized systems, Kubernetes has made service discovery something teams rarely build themselves, because it is built into the platform. The mechanism has a specific shape worth understanding.

When you create a Kubernetes Service, you get a stable virtual address, the ClusterIP, and a DNS name like payment.default.svc.cluster.local that never changes even as the pods behind it are created and destroyed.

Kubernetes continuously tracks which pods are healthy members of that Service through an Endpoints (or EndpointSlice) object, which is effectively the registry, maintained by the platform via third-party registration so your code registers nothing.

Calling code talks to the Service's DNS name, and CoreDNS resolves it to the ClusterIP. From there, kube-proxy (or the equivalent in a service mesh) load-balances the connection across the current healthy pods.

The piece that lets Kubernetes evict a dead pod is the readiness probe declared on the pod itself. It plays the same role as the health-check block in the Consul registration earlier: a URL the platform polls to decide whether the pod should receive traffic.

When a pod crashes or its /health endpoint starts failing, three failed probes mark it not-ready, Kubernetes pulls it out of the Service's Endpoints, and kube-proxy stops sending it connections. From the caller's side, the stable DNS name keeps working and traffic quietly shifts to the surviving pods, with no registration code anywhere in the application.

This is server-side discovery under a different name: the caller hits a stable address, and the platform does the registry lookup and load balancing on its behalf.

On Kubernetes, services find each other through Kubernetes Services and DNS, with no dedicated registry like Eureka involved. Outside Kubernetes, you need a dedicated registry, which is the next piece.

When the platform does not provide discovery, you run a purpose-built registry. Three names come up repeatedly, and the main difference between them is the consistency trade-off they make, which is a direct application of the CAP theorem to the registry itself.

Eureka, built by Netflix, deliberately chooses availability over consistency (an AP system).

During a network partition it keeps serving whatever instance data it has rather than refusing to answer, on the reasoning that a slightly stale list of instances is far better than no list at all when you are trying to route traffic. Each client caches the registry locally and can keep working even if the registry is briefly unreachable.

ZooKeeper and Consul lean the other way, toward consistency (CP systems), using a consensus protocol so every reader sees the same up-to-date view, at the cost of potentially rejecting reads during a partition to avoid handing out stale data.

Consul is the common choice for non-Kubernetes microservices today, pairing a CP core with rich health checking; ZooKeeper is older and often inherited rather than chosen.

The deciding factor is what you want to happen when the network breaks. A discovery system that picks consistency might briefly refuse to tell you where a service is; one that picks availability might tell you about an instance that just died.

For service discovery specifically, many teams favor the availability-first stance, because routing to an occasionally-stale instance and retrying beats being unable to route at all. That reasoning leads straight to the question of what happens when the registry has trouble.

The registry is the one component every discovery call depends on, which makes it the most dangerous single point of failure in the whole scheme.

If the registry goes down and nothing else changes, no service can find any other service, and the entire system seizes even though every individual service is healthy. A discovery design that ignores this leaves a gap at the center of the architecture.

Two defenses, used together, keep a registry outage from becoming a system outage. The first is to never run the registry as a single node: run it as a replicated cluster of several nodes so the loss of one does not lose the directory.

The second is client-side caching of last-known-good data. Each service caches the instance lists it has already fetched, so when the registry becomes unreachable, callers keep routing using their most recent snapshot instead of failing.

This is precisely the availability-first design Eureka is built around, including a "self-preservation mode" where the registry, on suddenly losing many heartbeats at once, assumes a network problem rather than a mass death of services and refuses to evict everyone, choosing stale entries over an empty registry.

The combination means a brief registry outage degrades freshness rather than function: routing continues on cached data and catches up when the registry returns.

One more issue shows up in real deployments, the split-horizon problem. The same service often needs different addresses depending on who is asking.

An internal caller in the same cluster should reach Payment by its private cluster address, while an external client coming through the edge must reach it by a public-facing address, and the registry has to serve the right answer to each.

Resolving this by hand requires separate records and conditional logic.

A service mesh handles it naturally by putting a proxy next to every service that knows the correct path for each direction of traffic, which is part of why meshes have become the common home for discovery in large systems, and the subject the next chapters build toward.

This worked example traces discovery end to end, using the registration and lookup from earlier. The Order service needs to call Inventory to reserve stock during checkout, and Inventory runs as several instances behind a client-side registry like Consul.

A new Inventory instance comes up at 10.0.7.4:8080. Once it is ready for traffic, it registers itself with the registry, sending its name, address, and a health-check URL, then renews its lease on a timer so the registry knows it is still alive.

Now an Order request arrives. The Order service queries the registry for healthy inventory instances, gets back a list that includes 10.0.7.4, picks it, and calls it directly to reserve stock. The reservation succeeds and checkout proceeds.

Then that instance crashes. It never gets to deregister, so for a moment its entry is still in the registry. The registry keeps polling , the calls now fail because nothing is listening, and after the critical check has been failing past the eviction window the registry removes 10.0.7.4 from the directory.

From that point on, every Order query for inventory returns only the surviving instances. New requests route around the dead address automatically, and no caller has to know an instance ever died.

The diagram traces the full life of the call. Steps 1 through 4 are the healthy path: the instance registers and heartbeats, Order looks it up, and the call goes through. Steps 5 and 6 are the failure: the crash stops the heartbeats, the failing health check tells the registry the instance is gone, and it is evicted. Steps 7 and 8 are the recovery: a later lookup returns only the survivors, so routing continues on healthy instances without any manual intervention. The registry's health-check loop is what turns a silent crash into a clean removal that callers never have to handle themselves.