AlgoMaster Logo

Design Unique ID Generator

Last Updated: December 25, 2025

Ashish

Ashish Pratap Singh

medium

Let’s begin by clarifying the requirements.

1. Clarifying Requirements

A system design interview should always start with a conversation to scope the problem. Here’s an example of how a candidate–interviewer discussion might flow:

After gathering the details, we can summarize the key system requirements.

1.1 Functional Requirements

  • Unique ID Generation: Generate globally unique IDs across distributed services. Each ID must fit within 64 bits.
  • Time Ordering: IDs should be K-sortable, meaning they roughly increase with creation time.

1.2 Non-Functional Requirements

  • High Availability: The service must be resilient to node failures. Aim for 99.99% availability.
  • Scalability: The system must scale to handle very high throughput (millions of IDs/sec).
  • Low Latency: ID generation should be extremely fast (e.g., <2ms).
  • K-Sortable: IDs should be roughly sortable by creation time.

2. Approaches to ID Generation

Now that we’ve clarified the requirements, let’s explore several approaches to generating unique IDs, evaluating how each performs across scalability, availability, ordering, and complexity dimensions.

Approach 1: Database Auto-Increment

The simplest way to generate unique IDs is by using a centralized database sequence or an auto-incrementing primary key.

In this setup, a single database server maintains a counter. Each time a client requests an ID, the database increments the counter and returns the new value, guaranteeing strict order and uniqueness.

This is commonly implemented with an AUTO_INCREMENT or IDENTITY column in SQL.

Every time a new record is inserted, the database automatically assigns the next sequential id value.

This creates a simple but problematic architecture where all services depend on a single database for ID generation.

While simple, this centralized approach comes with significant trade-offs.

Pros:

  • Guaranteed Uniqueness: The database enforces uniqueness by design.
  • Strict Ordering: IDs are monotonically increasing, which makes sorting and indexing straightforward.
  • Simplicity: Easy to implement and maintain — ideal for small-scale applications.

Cons:

  • Single Point of Failure (SPOF): If the database fails, the entire ID generation process stops.
  • Scalability Bottleneck: All ID requests go to one server, limiting horizontal scalability.
  • High Latency: Each ID request requires a network round trip to the central database.
  • Limited Throughput: The database can only handle a limited number of writes per second, which caps the rate of ID generation.

Adapting Auto-Increment for Distributed Systems

To overcome the limitations of a single database, you can adapt the auto-increment strategy for a multi-node environment using two common techniques.

1. Range-Based ID Allocation

In this model, each database node is pre-assigned a unique and non-overlapping range of IDs. This allows each node to generate IDs independently without communicating with others.

For example, in a three-node setup:

  • Node 1 can use IDs from 1 to 100000.
  • Node 2 can use IDs from 100001 to 200000.
  • Node 3 can use IDs from 200001 to 300000.
Limitations of Range-Based Allocation:
  1. Range Exhaustion: High-traffic nodes may exhaust their assigned range quickly, requiring reallocation or range expansion.
  2. Complex Management: As nodes are added or removed, reassigning and managing ranges can become complex.
  3. Waste of ID Space: Uneven traffic across nodes may leave some ranges underutilized.

2.Step-Based Auto-Increment

In this approach, each node generates IDs using a common step size equal to the number of nodes. Each node is also given a unique starting offset. This creates an "interleaving" pattern of IDs.

For example, if the step size is 3:

  • Node 1 starts at 1 and generates IDs: 1, 4, 7, 10, ...
  • Node 2 starts at 2 and generates IDs: 2, 5, 8, 11, ...
  • Node 3 starts at 3 and generates IDs: 3, 6, 9, 12, ...

This method ensures uniqueness but makes it difficult to add new nodes, as it would require changing the step size and offsets across the entire cluster.

Approach 2: UUID

A UUID is a 128-bit value designed to be unique across space and time without requiring any coordination between systems.

Unlike centralized or database-based approaches, UUIDs can be generated independently on any machine while still maintaining near-zero probability of collision.

There are multiple versions of UUIDs, but the two most commonly discussed are UUID v1 and UUID v4.

UUID v1 (Time + MAC-based)

UUID v1 combines a timestamp (down to 100-nanosecond precision) and the machine's MAC address.

This combination ensures global uniqueness since MAC addresses are unique per device, and timestamps prevent collisions within the same machine.

Example:

  • Time Component: 6ba7b810-9dad-11d1 (Represents a specific point in time)
  • Node Component: 00c04fd430c8 (Derived from the machine's MAC address)

UUID v1 is roughly time-sortable because it embeds the timestamp, but it also exposes the MAC address and generation time posing potential privacy risks.

UUID v4 (Random-based)

This is the most common version today. It's generated using 122 bits of cryptographically secure randomness. Only 6 bits are used for static version and variant information.

Example:

  • Version Info: The 4 in 41d4 indicates it's a v4 UUID.
  • Randomness: All other parts are generated randomly.

The probability of two UUID v4 values colliding is so infinitesimally small it's considered negligible for all practical purposes.

Pros

  • Simple and Decentralized: No coordination or central service is required. Every node can generate IDs independently.
  • No Network Latency: ID generation happens locally and instantly.
  • Collision-Free in Practice: Astronomically low probability of duplicates, even across billions of nodes.
  • Mature and Standardized: Supported natively in most programming languages and databases.

Cons

  • Not K-Sortable (v4): Randomly distributed IDs destroy index locality, causing fragmented B-tree indexes and slower inserts in databases.
  • Not Numeric: UUIDs are 128 bits and typically represented as 36-character strings, which take more space and memory to store and index.
  • Potential Security Risks (v1): Since UUID v1 encodes both the MAC address and timestamp, it can potentially expose system-level and temporal information.
  • Higher Storage and Bandwidth Overhead: Compared to compact 64-bit numeric IDs, UUIDs consume twice the space and are less cache-friendly.

UUID v7 (Time-based + Random)

UUID v7 is a new emerging standard that combines the time-ordering of Snowflake with the simplicity and decentralization of traditional UUIDs.

Structure of UUID v7:

  • 48 bits: Unix timestamp (in milliseconds)
  • 4 bits: Version (7)
  • 76 bits: Cryptographically secure random data

The leading timestamp ensures that IDs are chronologically sortable (K-sortable). The large random portion guarantees uniqueness with an extremely low probability of collision, even when millions of nodes are generating IDs in the same millisecond.

UUID v7 is poised to become the new standard for primary keys in distributed systems, but it's important to understand its trade-offs.

Pros:

  • K-Sortable by Default: Excellent for database index locality and write performance.
  • Fully Decentralized: No coordination or worker ID assignment is needed.
  • Simple Generation: Can be generated locally without network calls.
  • Globally Unique: The random component makes collisions practically impossible.

Cons:

  • Size: At 128 bits, it's twice the size of a 64-bit Snowflake ID, consuming more storage and index space.
  • Adoption: As an emerging standard, it is not yet natively supported in all databases or frameworks.

Approach 3: Ticket Server

The Ticket Server is a clever evolution of the centralized database model. Instead of hitting the database for every single ID, application servers fetch a block (or range) of IDs in one go and then dispense them from local memory.

This method was famously used by Flickr and later adopted by several large-scale systems before distributed ID generation algorithms (like Snowflake) became popular.

How It Works

The architecture introduces a dedicated, lightweight service (the Ticket Server) that acts as the single source of truth for ID blocks.

  1. A central Ticket Server manages the allocation of ID ranges.
  2. It uses a database for persistence, storing only a single counter for the next available ID.
  3. An Application Server requests a block of IDs (e.g., a block of 1,000) from the Ticket Server.
  4. The Ticket Server makes a single, atomic call to the database to reserve the next block.
  5. It returns the reserved range (e.g., [1001, 2000]) to the Application Server.
  6. The Application Server then serves IDs from this range in-memory, requiring no further network calls until its block is exhausted.

This architecture dramatically reduces the load on the database, as it's now only contacted once per block instead of once per ID.

The sequence diagram below shows the detailed interaction:

To atomically reserve a block, the Ticket Server can run a single atomic SQL command:

Pros

  • Numeric and K-Sortable: IDs are sequential, making them efficient for indexing and range queries.
  • Reduced Database Load: Only occasional requests are made to the database (once per block).
  • Low Latency: Most IDs are served directly from memory, eliminating per-request round trips.
  • Simple Scaling for Read-Heavy Workloads: By increasing block sizes or caching strategies, you can achieve very high throughput.

Cons

  • Single Point of Failure (SPOF): The Ticket Server becomes a bottleneck and a single dependency. If it goes down, no new IDs can be allocated.
  • Operational Complexity: Clients must handle local block exhaustion gracefully and fetch new blocks efficiently.
  • High Availability Setup is Non-Trivial: Making the ticket server highly available (e.g., using leader election or primary–secondary replication) adds configuration and coordination complexity.
  • Potential ID Gaps: If a server crashes before using up its allocated range, some IDs in that range are permanently lost (which may or may not be acceptable).

Approach 4: Snowflake-like Service

The Snowflake algorithm, originally developed by Twitter (now X), is a decentralized and highly scalable approach for generating globally unique, time-ordered 64-bit IDs.

It is widely considered the gold standard for distributed ID generation balancing uniqueness, ordering, performance, and fault tolerance without relying on a single coordination point at runtime.

How It Works

Each generated 64-bit ID is composed of multiple parts, each serving a specific purpose. The design cleverly encodes time, machine identity, and per-millisecond sequencing into a single compact number.

1. Sign Bit (1 bit)

  • Always set to 0 to ensure the generated number remains positive.
  • This allows the ID to fit cleanly within the signed 64-bit integer range used by most databases.

2. Timestamp (41 bits)

  • Represents the number of milliseconds since a custom epoch (e.g., 2020-01-01 00:00:00 UTC).
  • Using a custom epoch rather than the Unix epoch (1970) helps extend the usable lifespan.
  • A 41-bit timestamp can represent: 241milliseconds ≈ 69 years

3. Datacenter ID (5 bits)

  • Identifies the data center or region where the ID was generated.
  • Supports up to 2^5 = 32 data centers.
  • Helps maintain uniqueness in multi-region deployments.

4. Machine ID (5 bits)

  • Identifies the worker or node within the data center.
  • Supports 2^5 = 32 machines per data center.
  • Together, the datacenter and machine bits provide 2^10 = 1024 unique workers globally.

4. Sequence Number (12 bits)

  • A counter that increments for each ID generated within the same millisecond on the same machine.
  • Resets to 0 when the millisecond changes.
  • Supports up to: 212 = 4096 unique IDs per millisecond per machine.

This allows a single machine to generate millions of IDs per second with virtually no contention. Each field is bit-shifted and combined into a single 64-bit integer using bitwise operations.

Example (Pseudocode)

The result is a compact, globally unique, time-ordered ID.

Advantages

  • Globally Unique: No collisions across time, machines, or processes.
  • K-Sortable: IDs are roughly ordered by time, aiding database indexing and time-based queries.
  • High Throughput: Generates millions of IDs per second per node without coordination.
  • Low Latency: All logic runs locally — no network dependency.
  • Horizontally Scalable: Add more machines to increase capacity linearly.
  • Compact: Fits neatly within 64 bits (unlike UUIDs).

Potential Challenges

  • Clock Drift: If system clocks move backward, duplicate IDs can be generated.
    • Mitigation: Detect backward jumps and pause ID generation until the clock catches up.
  • Machine ID Coordination: Requires a reliable way to assign and track unique machine IDs.
    • Common solutions include Zookeeper, Etcd, or central registry services.
  • Time Dependency: When the 69-year window expires, a new epoch and migration strategy are needed.

3. Detailed Design

Now that we understand the Snowflake algorithm conceptually, let’s design the architecture and implementation details for a Snowflake-based ID generation service that can scale horizontally and operate reliably under real-world distributed conditions.

Architecture Overview

At a high level, the ID Generator Service consists of a cluster of stateless nodes. Each node can independently generate unique IDs once it acquires a unique worker ID during startup.

Worker ID Assignment

The most critical requirement for this design is ensuring that no two nodes share the same worker ID (10 bits total: 5 bits for datacenter + 5 bits for machine).

If two nodes accidentally share the same worker ID, they could produce duplicate IDs violating the system’s core guarantee.

We use a coordination service like ZooKeeper or etcd to assign unique worker IDs dynamically.

How It Works

  • On startup, each generator node creates an ephemeral sequential znode under a path such as /workers/id-.
  • ZooKeeper automatically appends a monotonically increasing sequence number, ensuring uniqueness.
    • Node 1 → /workers/id-0000000001
    • Node 2 → /workers/id-0000000002
    • Node 3 → /workers/id-0000000003
  • The node parses its sequence number and uses it as its worker ID.
  • If a node fails or disconnects, its ephemeral znode is automatically deleted, making its worker ID reusable.

This mechanism guarantees automatic registration, uniqueness, and fault recovery without human intervention.

Clock Synchronization

The timestamp component of the Snowflake ID makes the system sensitive to clock drift across servers.

If a node’s system clock moves backward, it could generate IDs that overlap with previously issued ones.

Example:
  • Suppose a node’s clock jumps backward by 50 ms.
  • The node now thinks it’s in an earlier time than the last generated ID.
  • If it continues generating IDs, they could collide with older ones, violating global uniqueness.

Solution:

1. NTP Synchronization

All nodes should run Network Time Protocol (NTP) daemons that synchronize their clocks with a trusted central source (e.g., a stratum 1 time server).

2. In-Service Detection

The generator should track the last timestamp used. If the current system time is less than the last recorded timestamp, it must:

  • Option 1 (Safer): Reject ID generation requests with an error, prompting clients to retry later.
  • Option 2 (Lenient): Spin-wait until the system clock catches up.
3. Trade-Offs:
  • The spin-wait method adds latency and can block threads.
  • The reject method offloads retry logic to the client but ensures no duplicate IDs are ever emitted.

In practice, rejecting is preferred in safety-critical systems, while spin-wait is acceptable in low-latency internal tools.

Implementation

Below is a simplified thread-safe implementation of the core Snowflake ID generator logic.

4. Summary and Wrap-Up

We designed a robust, distributed unique ID generator based on the Snowflake algorithm. This approach is highly performant and scalable because it requires no runtime coordination between nodes.

Key Takeaways:

  • Composition over Coordination: The ID's uniqueness comes from its structure (timestamp + worker_id + sequence) rather than from a central authority.
  • Worker ID is Critical: The most important design decision is how to reliably assign unique worker IDs. ZooKeeper is an excellent tool for this.
  • Time Matters: Relying on system clocks introduces challenges that must be handled gracefully, primarily through NTP and defensive service logic.

Quiz

Design Unique ID Generator Quiz

1 / 20
Multiple Choice

For a distributed unique ID generator, which requirement best describes K-sortability?