Imagine you're building an e-commerce application using microservices. Your system has several services, including an Order Service, Payment Service, Inventory Service, and Notification Service.

When a customer places an order, the Order Service needs to communicate with the Inventory Service to check product availability. In a simple application, you might configure the Inventory Service address directly:

This works as long as the service always runs at that address. But in a production microservices architecture, services rarely have fixed addresses. Instances are constantly being created, restarted, replaced, and scaled based on traffic.

For example, the Inventory Service might have three instances:

If one instance crashes, it might be replaced by another instance with a different IP address. Similarly, when traffic increases, additional instances might be launched automatically.

How does the Order Service know which Inventory Service instances are currently available? Hardcoding IP addresses isn't practical because those addresses change frequently. This is the problem that Service Discovery solves.

Service Discovery is the mechanism that allows services to dynamically find and communicate with other services in a distributed system. Instead of storing fixed IP addresses, services use logical names such as:

The discovery mechanism maps these names to available service instances. For example:

When the Order Service needs to contact the Inventory Service, it uses service discovery to locate an appropriate destination. This allows service instances to be added, removed, or replaced without requiring changes to the calling service. Service discovery is especially important in environments that use containers, orchestration platforms, and automatic scaling.

There are two main approaches:

The primary difference is who is responsible for selecting the destination service instance.

Before understanding the two approaches, we need to understand the Service Registry.

A Service Registry is a component that maintains information about available service instances. It typically stores information such as:

For example:

The registry allows clients or infrastructure components to discover currently available instances. Common technologies used for service discovery include Consul, Netflix Eureka, and Kubernetes service discovery.

How Do Services Register?

There are two common registration approaches.

Self-Registration

Each service instance registers itself with the registry when it starts. It also periodically sends heartbeats or renews its registration to indicate that it is still alive. When the instance shuts down, it deregisters itself.

Third-Party Registration

An external component manages registrations on behalf of service instances. For example, a container orchestration platform can track which instances are running and update the discovery system accordingly. This keeps registration logic outside individual applications.

In either case, the registry needs a way to remove unhealthy or terminated instances so callers don't continue sending traffic to unavailable destinations.

In Client-Side Service Discovery, the calling service is responsible for finding available instances and selecting which instance should receive a request. The client communicates with the Service Registry to retrieve the available destinations. It then selects an instance using a load-balancing strategy.

Consider an Order Service calling an Inventory Service.

How It Works

Step 1: Service registration

Inventory Service instances register their network addresses with the Service Registry.

Step 2: Service lookup

The Order Service queries the registry for available Inventory Service instances. The registry returns a list of healthy destinations.

Step 3: Instance selection

The Order Service selects one instance using a load-balancing algorithm. For example, it might use round-robin to distribute requests across available instances.

Step 4: Direct communication

The Order Service sends the request directly to the selected Inventory Service instance. The registry isn't involved in forwarding the actual request. The client may also cache the discovered instances and periodically refresh the list rather than contacting the registry for every request.

Advantages

Fewer infrastructure hops: Requests travel directly between services without requiring a separate routing proxy.

Flexible load balancing: Each client can implement load-balancing strategies appropriate for its requirements.

Better routing control: Clients can potentially select instances based on availability zones, latency, or other metadata.

Disadvantages

More client complexity: Every calling service needs discovery and load-balancing functionality.

Language-specific implementations: Applications written in Java, Go, Python, or Node.js may need different client libraries.

Stale instance information: Clients might temporarily use outdated addresses if registry updates haven't reached them.

Client-side discovery works well when teams have consistent technology stacks and can manage discovery through shared libraries.

In Server-Side Service Discovery, the calling service doesn't need to know which backend instances are available. Instead, it sends requests to a stable address provided by a load balancer, proxy, or routing component. That intermediary is responsible for selecting a destination. For example:

How It Works

Step 1: Service registration

Inventory Service instances are registered with the discovery system.

Step 2: Request submission

The Order Service sends a request to a stable logical endpoint. For example:

The client doesn't need to choose an individual Inventory Service instance.

Step 3: Instance resolution

The routing infrastructure obtains information about available instances from the registry or orchestration platform.

Step 4: Request forwarding

The proxy or load balancer selects a suitable instance and forwards the request. The response is then returned to the Order Service.

Advantages

Simpler clients: Applications don't need to implement their own service discovery and load-balancing logic.

Language independence: Services written in different programming languages can use the same routing infrastructure.

Centralized traffic management: Routing policies, health checks, and traffic distribution can be managed consistently.

Disadvantages

Additional infrastructure: The system requires routing components that must be deployed, scaled, and monitored.

Extra network processing: Requests pass through an intermediary, which can introduce additional latency.

Potential bottleneck: A poorly designed or undersized routing component can affect availability and throughput.

These risks can be reduced by using redundant, horizontally scalable routing infrastructure.

Kubernetes provides a practical example of service discovery in modern microservices architectures. Suppose you deploy three replicas of the Inventory Service. Each replica runs inside a Pod with its own IP address. These Pod IP addresses can change whenever Pods are restarted or replaced.

Kubernetes solves this using a resource called a Service. A Kubernetes Service provides a stable logical endpoint for a changing set of backend Pods. For example, the Order Service can call:

Kubernetes DNS resolves the Service name to its stable Service address. The cluster's service-routing infrastructure then directs traffic to one of the eligible backend Pods.

Behind the scenes, Kubernetes tracks the available Pods through its control plane and EndpointSlices. The routing implementation may use kernel networking mechanisms or eBPF, depending on the cluster.

This is an example of infrastructure-managed, server-side discovery.

Kubernetes also supports headless Services, which can return individual Pod addresses through DNS. These are useful when applications need more direct control over endpoint selection.

Service discovery must handle more than simply finding IP addresses. Consider what happens when an Inventory Service instance crashes. If the discovery system continues advertising its address, requests might fail. A reliable discovery system needs mechanisms for identifying unavailable instances.

Health Checks and Readiness

Health checks help determine whether a service instance should receive traffic. For example, an instance might be running but unable to handle requests because its database connection is unavailable. A readiness check can identify this condition and remove the instance from the set of eligible destinations.

Registration Expiration

Some registries use leases or heartbeats. If an instance fails to renew its registration within a configured period, the registry can expire that registration. This helps remove terminated instances, although failure detection isn't instantaneous.

Graceful Shutdown

When an instance is being terminated, it should stop receiving new requests before shutting down completely. This allows existing requests to finish and reduces errors during rolling deployments.

Stale Discovery Data

Even with health checks, clients or proxies may temporarily use outdated information. For example, a client might select an instance that became unavailable moments earlier. Therefore, service discovery should be combined with timeouts, appropriate retries, and connection-failure handling.

Service discovery and load balancing are closely related, but they solve different problems.

For example, a Service Registry might return three Inventory Service instances. A load balancer then chooses one of those instances using round-robin, least connections, or another strategy.

Client-side discovery usually combines discovery and load balancing inside the calling application. Server-side discovery delegates instance selection to routing infrastructure. Understanding this distinction is important because the two concepts are closely related.

When designing service discovery for production, consider the following principles.

Avoid hardcoded service addresses. Use stable logical names so deployments and scaling don't require application changes.

Use health-aware discovery. Route traffic only to instances that are ready to handle requests.

Make the registry highly available. Discovery infrastructure shouldn't depend on a single registry instance.

Cache discovery information carefully. Caching reduces lookup overhead, but stale entries must be refreshed or invalidated.

Handle failures at the caller. Use timeouts, controlled retries, and circuit breakers where appropriate.

Secure service discovery. Protect registry access and ensure only authorized components can register or update service information.

Use platform-native capabilities when possible. If Kubernetes already provides suitable discovery and routing, introducing a separate discovery platform may add unnecessary complexity.

Service discovery is a fundamental building block of microservices architectures. It allows services to find one another dynamically without relying on fixed IP addresses or manually maintained configurations.

The two main approaches differ in where instance selection happens. In client-side discovery, the calling application discovers and selects a destination. In server-side discovery, a proxy or load balancer performs that responsibility on behalf of the caller.

Both approaches allow services to scale, restart, and move without breaking communication. The key takeaway is that service discovery separates a service's logical identity from the physical location of its running instances, making dynamic microservices environments easier to manage and operate.