Calling a function inside one process and calling a service across a network can look deceptively similar:

The first call transfers control to code in the same process. The second may serialize data, wait for a connection, cross several network devices, enter a remote queue, run in another process, and send a response back along a path that can behave differently from the outgoing path.

The similar syntax hides a radically different failure model.

The fallacies of distributed computing are eight assumptions that developers often make when they treat remote communication like a local function call:

They are called fallacies because none is a safe general assumption. A particular call may be fast, secure, and successful, but software must remain correct when those favorable conditions stop holding.

These ideas are not merely warnings about rare outages. They shape API boundaries, latency budgets, payload sizes, security controls, deployments, observability, and ownership. They explain why code that works perfectly on a laptop can become slow or ambiguous in production.

A local function call happens within one failure domain. If the process crashes halfway through the call, both caller and callee disappear together. Memory access is fast, data structures do not need a wire representation, and the caller receives either a return value or a local exception while the process remains alive.

A remote call spans at least two processes and a communication path between them:

Every stage has state, queues, resource limits, and its own owner. The request can fail before reaching the server, while the server is processing it, or after the server has completed the work but before the response reaches the caller.

This creates an outcome that normal local calls rarely expose: unknown.

Suppose an order service sends a request to reserve an item and receives no response. The reservation may not have happened, may still be running, or may have completed successfully while only the response was lost. The caller cannot determine the business outcome from silence alone.

The eight fallacies are different views of this network boundary. Understanding them begins with refusing to treat that boundary as invisible.

The first fallacy assumes that a request sent will arrive and that its response will return.

Real networks make a best effort to move packets through many independent components. Packets can be dropped by a congested queue, a faulty link, an overloaded host, or a filtering device. Routes can temporarily disappear. A server process can restart while a request is in flight. A proxy can close an idle connection, and a client can lose connectivity after sending data.

Reliable transports such as TCP reduce this uncertainty. TCP detects loss, retransmits bytes, removes duplicates, and delivers an ordered byte stream while a connection remains usable. That is valuable, but it does not make an entire distributed operation reliable.

TCP cannot guarantee that:

Consider a request that creates order 8472:

From Checkout's perspective, this exchange looks similar to a request that never reached the Order service. In one case, sending again may be necessary. In the other, sending again may create a duplicate unless the operation has an identity and duplicate-safe semantics.

This is the central lesson of the reliability fallacy: communication failure and operation failure are not the same fact.

A production caller must put a bound on how long it waits, expose network failures instead of inventing a successful result, and account for an unknown outcome. Operations with side effects should carry stable request identities or otherwise define what repeated delivery means. Those measures do not make the network perfectly reliable; they make the application honest about uncertainty.

A local function may complete in nanoseconds or microseconds. A remote call commonly takes milliseconds, and its slowest executions can take much longer. Replacing a function call with a service call changes both the scale and variability of waiting.

The elapsed time of a remote call includes more than propagation across a cable:

Some components may be absent on a reused connection, while others vary from one request to the next. Queueing is particularly important: a server can execute an operation in 5 milliseconds when idle but return it in 300 milliseconds after the request waits behind other work.

Round Trips Accumulate

Suppose an API handler makes four dependent service calls. Each normally takes 25 milliseconds, and the next cannot begin until the previous one completes:

That is before the handler's own work, the client's connection to the API, or any slow outlier. A design with twenty small sequential calls can be unusable even when every individual service appears fast.

Independent calls can sometimes run concurrently, but concurrency does not make latency disappear. The overall request must often wait for its slowest required dependency. As fan-out grows, the probability that at least one dependency call lands in its slow tail also grows.

For example, if a request needs successful responses from 30 replicas or partitions, an unusually slow response from any required participant can determine the user's latency. Average latency alone hides this behavior. Production systems therefore care about percentiles such as p95 and p99 as well as the mean.

Physical Distance Still Matters

Signals travel quickly, but not instantaneously. Long routes, intermediate devices, and protocol round trips make communication between distant regions fundamentally slower than communication inside one data center. No software abstraction can remove propagation time.

Applications should minimize unnecessary network round trips, avoid chatty interfaces, and assign each remote operation a realistic share of the end-to-end latency budget. Measurements should cover the full latency distribution under representative load. A remote call can be inexpensive enough for a design, but it is never literally free of waiting.

Bandwidth is the rate at which a path can carry data. Every link, network interface, proxy, and host has finite capacity, and that capacity is shared.

An oversized payload may look harmless during development because there is one client on a fast local network. At production traffic levels, the same payload becomes a capacity problem.

Suppose an endpoint returns a 40 KB response at 4,000 responses per second:

That estimate covers payload bytes in one direction. Request data, protocol overhead, replication, internal service calls, and retransmitted data add more traffic. A nominal 1 Gb/s interface cannot sustain a 1.28 Gb/s payload rate, and a faster interface may merely move the bottleneck to a proxy or another shared link.

Finite bandwidth affects latency before a link reaches an obvious hard limit. Bursts build queues. Requests wait behind large transfers. When buffers fill, packets are dropped, which causes reliable transports to retransmit and reduce their sending rate. One service's bulk traffic can therefore slow unrelated latency-sensitive traffic sharing the same path.

Bandwidth also varies across callers. A service tested on a data-center link may be consumed by mobile clients, branch offices, VPN users, or services in another region. Payload sizes that are trivial on one path may dominate response time on another.

Practical design begins by counting bytes as well as requests. Return the fields a caller needs, paginate large collections, stream data when the consumer can process it incrementally, and compress suitable content when the CPU trade-off is worthwhile. Capacity monitoring should include bytes per second and packet rate, not just request rate.

The important distinction from the latency fallacy is that latency concerns how long an exchange takes, while bandwidth concerns how much data a path can carry per unit time. A path can have high bandwidth and high latency, or low bandwidth and low latency. Neither metric substitutes for the other.

A remote request leaves the caller's process and crosses components the application does not fully control. Assuming that traffic is trustworthy because it is “inside the network” turns network location into a security boundary it was never designed to be.

Threats are not limited to someone physically tapping an internet cable. Credentials can leak. A compromised workload can send traffic from an internal address. A routing or name-resolution mistake can direct a client toward the wrong endpoint. Shared infrastructure can be misconfigured. An authorized service can also call an operation it should not be allowed to perform.

Secure communication must answer separate questions:

Confidentiality: Can an observer read the data?

Integrity: Can data be modified without detection?

Authentication: Does each endpoint know the identity of the other?

Authorization: Is that authenticated identity allowed to perform this operation?

Encryption provides confidentiality and normally protects integrity, but encryption alone does not establish that a caller is permitted to issue a refund. Authentication establishes identity, but an authenticated service can still lack authorization for a particular resource.

A private subnet, VPN, firewall, or network policy can reduce exposure. These are useful layers, not replacements for authenticated and encrypted application communication. Sensitive operations should verify identity and permission at the service boundary. Applications should also validate remote input, because data received from another service remains external input to the receiving process.

The security fallacy changes a basic engineering question. Instead of asking, “Can this address reach my service?” ask, “Which authenticated identity is making this request, what is it allowed to do, and how are the request and response protected in transit?”

Network topology is the arrangement of endpoints and the paths connecting them. Distributed applications often behave as if a hostname always maps to the same machine, a connection always reaches the same process, and today's route will exist tomorrow.

Production topology changes constantly:

Even planned changes are visible to applications. An existing TCP connection continues to refer to its current endpoints; changing a DNS answer does not redirect that established connection to a new server. A client may also cache an earlier name resolution result after the corresponding instance has been removed.

Hardcoded IP addresses turn ordinary replacement into an outage. Assuming that a pooled connection remains valid forever turns a routine deployment into connection errors. Treating one route as permanent makes regional failover brittle.

Applications should address logical services rather than individual processes, refresh endpoint information, tolerate connection replacement, and expect membership to change while requests are active. Deployments should remove instances from new traffic before terminating their active work, while clients should be able to reconnect without assuming that they will reach the same process.

Topology change is not exceptional behavior to be tested only during a disaster exercise. Every rolling deployment is a topology change.

The word administrator refers to whoever controls a part of the distributed system. Even within one company, no single person or team usually controls the caller, server, network, name resolution, load balancer, certificates, cloud platform, and every dependency.

An apparently simple service call might cross:

Each group has its own deployment schedule, permissions, objectives, and operational tools. A firewall rule can change without an application release. A certificate policy can become stricter while an old client is still deployed. A service team can reduce its request limit without knowing that a batch system depends on the former value.

The problem becomes more obvious across organizations. Public APIs, payment providers, identity providers, and cloud services are operated independently. Their maintenance, incident response, and compatibility decisions do not occur under the caller's control.

This fallacy is not an argument for putting every component under one team. It is a reminder that coordination is part of the design.

Remote interfaces need explicit contracts, versioning expectations, ownership, and change procedures. Telemetry should identify which boundary failed rather than reporting only “network error.” Operational documentation should make it possible to find the responsible owner without relying on personal knowledge.

When several administrators are involved, a correct system must tolerate reasonable independent change. It cannot depend on everyone deploying the right configuration at exactly the same moment.

Bandwidth is only one cost of moving data. A remote exchange consumes resources even when the network has abundant spare capacity.

Before transmission, application objects are serialized into bytes. Data may be compressed and encrypted. The runtime and kernel allocate buffers, copy data, manage sockets, schedule work, and process protocol state. The receiver performs the corresponding work in reverse. Long-lived connections consume file descriptors and memory even while idle.

The path itself may involve load balancers, gateways, service proxies, network-address translation, logging, and cross-region links. Cloud providers may charge for egress, inter-zone traffic, load-balancer processing, or public addresses. Operators also pay for the capacity needed to handle peak rather than average traffic.

Consider an external API that handles 10,000 requests per second and makes twelve internal calls for each request:

Even small internal messages now require 120,000 serializations, connection-level exchanges, server dispatches, and response decodings every second. If a boundary is introduced merely to make code organization look cleaner, its operational cost may outweigh its architectural value.

This fallacy is distinct from finite bandwidth:

A network boundary should therefore be intentional. Coarse-grained APIs, connection reuse, sensible batching, caching, and avoiding redundant transfers can reduce transport work. The objective is not to eliminate remote calls; it is to make each one earn its cost.

A homogeneous network would give every endpoint the same protocol support, address family, path capacity, packet-size limits, security policy, and behavior. Real distributed systems mix all of these.

A single application may communicate across:

Two paths to the same service can therefore behave differently. A request may succeed from one availability zone and fail from another. Small messages may work while larger ones disappear because a path handles packet sizes differently. A persistent connection may survive in a data center but be closed after a short idle period by a carrier or enterprise gateway.

Application heterogeneity matters too. During a rolling deployment, new servers must often communicate with old clients, and new clients with old servers. A field, protocol feature, or security setting supported by one side may be unknown to the other. “All services use HTTP” does not make the environment homogeneous; versions, intermediaries, limits, and interpretations can still differ.

Standards reduce these differences by defining shared behavior, but standards do not guarantee identical implementations or configurations.

Robust systems negotiate optional capabilities where appropriate, use explicit and compatible message formats, set conservative size limits, and test more than the fastest production-like path. Metrics should be separable by region, network, protocol version, client type, and software version; an overall success rate can hide a complete failure for one minority group.

The eight assumptions rarely fail in isolation.

Imagine that an Inventory service becomes slow for requests from one availability zone because of a path change. The service is still reachable, but calls take longer.

The result can be a feedback loop:

This is why distributed-system incidents often look larger than their initiating fault. One slow path can become resource exhaustion across several services. The fallacies provide a vocabulary for tracing that expansion instead of labeling the whole event “the network was down.”

An API schema describes valid messages, but a production remote call needs a broader contract. Before making a dependency part of a request path, engineers should be able to answer:

For example, “call the Pricing service” is incomplete. A useful contract is closer to:

The exact values depend on the system. The important practice is making them explicit. Hidden assumptions become production surprises; explicit assumptions can be measured and tested.

Development environments often make every fallacy appear true. Client and server run on one laptop, latency is tiny, bandwidth is plentiful, there is one developer, addresses remain fixed, traffic never leaves the machine, and every component uses the same software version.

The application has not proved that its network behavior is correct. It has tested a particularly favorable topology.

Useful testing introduces the properties that localhost hides:

The goal is not to reproduce every possible outage. It is to verify that the application exposes uncertainty, bounds resource use, and remains diagnosable when a remote call stops behaving like a local one.

The fallacies do not mean every network call will fail. They mean success cannot be assumed for every call and every environment.

TCP reliability does not make a business operation reliable. TCP protects an ordered byte stream while the connection is viable; it cannot prove that the remote application completed an operation.

A timeout does not prove that the server did nothing. The request or response may have been delayed or lost after the operation took effect.

Low average latency is not zero latency. Sequential calls and slow-tail events can dominate end-to-end response time even when the mean looks healthy.

A faster link does not make bandwidth infinite. Capacity remains finite and shared, and bursts can fill queues before average utilization looks high.

More bandwidth does not make transport free. Serialization, encryption, buffers, connections, infrastructure, and data transfer still consume resources and money.

An internal address does not establish trust. Identity, confidentiality, integrity, and authorization require deliberate controls.

A DNS name does not make topology static. Its answers can change, caches can retain older answers, and existing connections keep their current endpoints.

One company does not imply one administrator. Teams, providers, and platforms independently control different parts of the request path.

Using standard protocols does not make a network homogeneous. Paths, intermediaries, versions, limits, and endpoint capabilities still differ.

The eight fallacies are not eight independent checkboxes. One incorrect assumption can amplify the consequences of several others.

The eight fallacies expose unsafe assumptions behind remote calls. Networks can lose connectivity or responses and leave outcomes ambiguous, latency is nonzero and variable, and bandwidth is finite and shared. Communication is not inherently secure, so identity, authorization, confidentiality, and integrity require explicit design.

Topology changes during deployments, scaling, failures, and client movement. Multiple teams and providers administer the path, transport consumes finite resources and money, and networks remain heterogeneous across devices, protocols, policies, and software versions.

A remote call is not a local function with a longer name; its timing, outcome, path, cost, and trust must all be designed explicitly.