Premium Video

This video is available to premium subscribers only

Unlock Full Access

Networks fail, responses get lost, clients retry, and message queues may deliver the same message more than once. As a result, the same request can reach a system multiple times. If the system processes every retry as a new request, a payment could be charged twice or an order could be created twice.

This lesson covers why duplicate requests and messages are so common in distributed systems, how to make operations safe to repeat, and the main techniques used to detect and remove duplicates.

It also covers how to approach deduplication in a system design interview, since it comes up any time a system retries requests or processes messages from a queue.