This problem is a common choice in system design interviews and serves as a great starting point to practice designing scalable, high-traffic systems.

In this chapter, we will explore the high-level design of a URL shortener.

Lets start by clarifying the requirements:

1. Clarifying Requirements

Before starting the design, it's important to ask thoughtful questions to uncover hidden assumptions, clarify ambiguities, and define the system's scope more precisely.

Here is an example of how a discussion between the candidate and the interviewer might unfold:

After gathering the details, we can summarize the key system requirements.

1.1 Functional Requirements

1.2 Non-Functional Requirements

2. Back-of-the-Envelope Estimation

To understand the scale of our system, let’s make some reasonable assumptions.

Writes

Reads

Storage (1 year)

Each link stores:

Total ≈ 300 bytes per link

Annual storage: 10M links/day * 365 days * 300 bytes ≈ 1.1 TB for the link table per year.

3. Core APIs

The URL shortener service needs a minimal but powerful set of APIs to create, manage, and resolve shortened links. Below are the two core APIs required for the basic functionality.

1. Create Short URL

Endpoint: POST /shorten

This endpoint takes a long URL (and optional parameters) and generates a corresponding short URL.

Sample Request:

Sample Response:

Error Cases:

2. Redirect to Long URL

Endpoint: GET /{short_url_key}

This endpoint takes the short URL key and redirects the user to the corresponding long URL.

Sample Response:

Behavior:

4. High-Level Design

At a high level, our system must satisfy two core requirements:

Since the system has a very high read-to-write ratio (billions of reads vs. millions of writes), it is beneficial to separate the URL generation service from the redirection service. This separation allows each service to be scaled independently based on its traffic pattern.

Core Components

1. Clients

Clients are end-users or applications interacting with the system via web browsers, mobile apps, or third-party integrations (e.g., APIs).

They use:

2. Load Balancer

The Load Balancer sits in front of all application servers and plays a key role in ensuring high availability and scalability.

Responsibilities:

3. URL Generation Service

This service handles all write operations, including creation and management of new short URLs.

Key Responsibilities:

Since write traffic is relatively low compared to reads, this service doesn’t need to scale aggressively.

4. Redirection Service

The Redirection Service handles the overwhelming majority of traffic — typically billions of GET requests per day. It must be ultra-fast, stateless, and horizontally scalable.

Responsibilities:

This component’s design should focuse on low latency, caching efficiency, and high read throughput.

5. Database

The database stores the mapping between short codes and long URLs, along with relevant metadata.

Must support:

High-Level Flow

1. Creating a Short URL

2. Accessing a Short URL

5. Database Design

5.1 SQL vs NoSQL

To choose the right database for our needs, let's consider some factors that can affect our choice:

Given these points, a NoSQL database like DynamoDB or Cassandra is a better option due to their ability to efficiently handle billions of simple key-value lookups and provide high scalability and availability.

5.2 Database Schema

Even though our system is simple, we should carefully design the schema to support both redirection and user management. At a minimum, we need two core tables.

1. URL Mappings Table

Stores the relationship between a short code and its corresponding long URL.

2. Users Table

Stores user-related information for personalization, analytics, and management.

6. Design Deep Dive

Now that we have the high-level architecture and database schema in place, let’s dive deeper into some critical design choices.

6.1 Unique URL Generation

The short code generation strategy is one of the most critical aspects of a URL shortener. A good algorithm should ensure that the generated codes are:

Let's explore three primary approaches, moving from simple to highly scalable.

Approach 1: Hashing + Encoding (Deterministic)

This is one of the simplest and most popular techniques for generating short URLs.

The idea is to use a cryptographic hash function to create a unique fingerprint of the long URL, then encode it into a compact, URL-safe string.

How It Works

1. Canonicalize the URL

Before hashing, the URL must be standardized to avoid generating multiple short links for what is essentially the same destination.

Example: The following URLs all point to the same page but would hash differently if not normalized:

Canonicalization Steps:

After canonicalization, these all become:

2. Generate a Hash

Apply a cryptographic hash function like SHA-256, SHA-1, or MD5 to the canonicalized URL.

For example:

This gives us a fixed-length fingerprint (128 bits for MD5, 256 bits for SHA-256).

3. Truncate and Encode

We don’t need the full hash, it’s too long for a short link.

Instead, we take the first few bytes (e.g., 48 bits = 6 bytes) and encode them using Base62.

Example Workflow:

The specific choice of 6 bytes (48 bits) is important because it produces a decimal number that typically converts to a Base62 string of approximately 7 characters.

This method is deterministic. The same long URL will always produce the same short code.

This means:

Collision Risks

However, hash collisions are possible because we truncate the hash.

Two different URLs could (rarely) produce the same short code:

→ Might both hash to → DZFbb43

Even though cryptographic hash collisions are rare, truncation reduces the available bit space, making collisions statistically inevitable at scale.

Collision Resolution Strategies

When a newly generated short code already exists in the database (detected via a UNIQUE constraint violation), we need a fallback strategy.

1. Re-hash with a Salt

Add a random salt or nonce to the original URL and hash again: hash = sha256(url + salt)

Repeat until a unique short code is found.

2. Append a Suffix

Attach a small counter or suffix to the colliding short code:

Both methods eliminate duplicates but come with trade-offs.

Trade-off: Both resolution strategies negate the stateless benefit by requiring at least one database lookup, adding latency and complexity to the write path.

Approach 2: Global Counter (Non-Deterministic)

This is one of the simplest and most reliable ways to generate short URLs.

Instead of hashing, it relies on a global, monotonically increasing counter to ensure every short code is unique by design.

How It Works

The entire system revolves around a centralized counter service that generates the next available integer ID in sequence and never repeats it.

1. Request a Unique ID

When an application server receives a request to shorten a URL, it contacts a Counter Service typically implemented with a fast in-memory data store like Redis, etcd, or Zookeeper.

Redis provides the INCR command, which is atomic by nature.

Each time this command is called, Redis:

Atomicity ensures that even under extreme concurrency, no two servers can ever get the same number.

2. Encode the ID

The application server receives the unique integer ID (e.g., 123456789) from the counter service. It then applies Base62 encoding to this number to create the final, compact short code.

Example Growth:

Even after generating a billion short URLs, the code length stays under 6–7 characters, which is extremely efficient.

End-to-End Example

Pros

Cons

Addressing the Limitations

To scale and secure this approach, modern URL shorteners extend it in two key ways.

Sharded Counters

Instead of one global counter, deploy multiple counters, each responsible for a subset of IDs.For example, 1024 counters distributed across regions or shards.

Each ID combines:

Formula: global_id = (shard_id << 20) | local_counter

This gives each shard up to 2²⁰ (≈1 million) IDs before wraparound.

Benefits:

Trade-off: Slightly more complexity in managing shards and ensuring IDs don’t overlap.

ID Obfuscation

To prevent sequential IDs from being easily decoded shuffle bits using a simple Feistel cipher or XOR mask.

The transformation is reversible (so you can still decode IDs if needed). This produces short codes that appear random even though they are derived from sequential IDs.

This adds privacy without sacrificing performance.

Approach 3: Distributed unique ID generator

This is the industry-standard solution for generating unique, non-deterministic IDs at a massive scale. It's a hybrid approach that avoids the need for a centralized counter.

How It Works

Each ID is a 64-bit integer made up of multiple components that together guarantee uniqueness even across thousands of machines.

1. Timestamp (41 bits)

This represents the number of milliseconds since a custom epoch (a chosen start date, e.g., Jan 1, 2020). Using a custom epoch instead of the Unix epoch extends the lifetime of the system.

41 bits can store 2^41 milliseconds, approximately 69.7 years of continuous operation.

So if your epoch starts in 2025, you’ll have ID space until roughly 2094.

2. Worker ID (10 bits)

Each worker (application server or service instance) gets a unique identifier at startup.

Worker ID Assignment: When a worker starts up, it connects to a coordination service like ZooKeeper or etcd. It registers itself and requests a unique worker ID, which it holds for its lifetime or for a set "lease" period. This startup coordination is vital to prevent two workers from accidentally using the same ID.

3. Sequence Number (12 bits)

This is a local, in-memory counter that tracks how many IDs the worker has generated within the same millisecond.

If a worker hits 4,096 requests within one millisecond, it must wait for the clock to tick to the next millisecond before continuing.

Pros:

Cons:

Summary and Recommendation

6.2 Fast URL Redirections

The redirection service is the most performance-critical component of a URL shortener.

Every time a user clicks a short link, this service must:

Even tiny delays (just a few milliseconds) can degrade user experience at scale, especially when serving billions of redirects per day.

Choosing the Right Redirect Code

Before optimizing performance, we must decide how to perform the redirect.

The choice between 301, 302, and 307 affects both speed and control.

The Read Path

To achieve low latency globally, we design a "waterfall" lookup path. The system tries to find the long URL in the fastest, closest cache available. It only proceeds to the next, slower layer on a cache miss.

Layer 1: Browser Cache

If a 301 redirect was previously issued, the browser already knows the final URL. The request never reaches your servers, the user is instantly redirected.

Layer 2: CDN / Edge Cache

For global services, a Content Delivery Network (CDN) is essential. CDNs like Cloudflare, Akamai, or Fastly maintain edge servers near users worldwide.

Cache Hit: The CDN instantly serves the cached 302 response with the long URL (≈10–50 ms latency).

Cache Miss: The CDN forwards the request to your origin server.

Layer 3: In-Memory Cache (Redis/Memcached)

At your application layer, the first lookup happens in a distributed in-memory cache.

Cache Hit: The application server immediately returns the long URL. The CDN then caches this response for future users (cache-aside pattern).

Cache Miss: If not found in Redis, the application proceeds to the final layer, the database.

Layer 4: Database (Source of Truth)

The database holds the authoritative mapping of short codes to long URLs. Since lookups are simple key-value reads, NoSQL stores like DynamoDB, Cassandra, or ScyllaDB are ideal.

Query Example:

Once retrieved:

6.3 Supporting Custom Aliases

Custom aliases allow users to create human-readable short codes instead of system-generated ones.

For example:

This feature is especially valuable for marketing campaigns, social media sharing, and brand consistency, where memorability and aesthetics matter.

However, supporting custom aliases introduces challenges around validation, uniqueness, and race conditions that must be carefully handled to maintain system integrity.

1. API Design

The API for link creation should accept an optional custom_alias parameter along with the long URL.

If the custom_alias field is not provided, the service falls back to auto-generating a short code using the ID generation algorithm.

2. Rigorous Validation

Before creating a custom alias, the system must validate it against strict rules to prevent conflicts, misuse, or routing errors.

All checks should occur before any database operation to minimize load on backend systems.

3. Ensuring Uniqueness and Handling Race Conditions

This is the most critical aspect of custom alias creation.

The Race Condition Problem

Two users could try to claim the same custom alias (summer-sale) at the exact same moment.

Without proper safeguards, both requests might initially think the alias is free leading to duplicate entries or inconsistent state.

The Reliable Solution

Instead of relying on application logic, use the database itself as the source of truth by defining a UNIQUE constraint on the alias column.

Schema Example:

6.4 Supporting Link Expiration

Link expiration is essential for maintaining both security and data hygiene.

It is commonly used in:

Without proper expiration handling, a URL shortener’s database could grow endlessly, degrading performance and increasing storage costs.

There are two primary ways to handle expired links: Active Deletion and Passive Expiration.

1. Active Deletion (Background Job)

In this model, a scheduled worker process periodically scans the database for expired links and deletes them.

Example:

Typically, this runs every hour or day using a background job scheduler such as Cron, Celery, or Airflow.

Pros

Cons

2. Passive Expiration (Real-Time Check)

In this approach, the link is never physically deleted when it expires.

Instead, the application layer checks the expiration timestamp during the redirect lookup.

Logic:

Pros

Cons

3. Recommended Hybrid Approach

A hybrid strategy combines the precision of passive expiration with the efficiency of occasional cleanup.

Step 1: Passive Expiration for Read Path

Step 2: Lazy Cleanup for Data Hygiene

4. Caching and Expiration Consistency

A crucial design consideration is ensuring all cache layers respect link expiration.

If not handled carefully, caches might continue serving redirects after the link has expired.

Problem Example

Solution

When writing to any cache, set the cache TTL to the smaller of:

TTL_cache = min(link.remaining_lifetime, default_cache_ttl)

This ensures cache TTLs align with link lifetimes to prevent expired redirects from being served.

6.5 Analytics – Click Count

A modern URL shortening system is not complete without analytics. The most fundamental metric is click count — the number of times each shortened link is accessed.

This data powers insights such as:

However, counting clicks accurately at scale introduces challenges related to performance, consistency, and real-time aggregation.

What Is a "Click"?

A click is registered whenever a user follows a shortened link that results in a successful redirect to the target URL.

To prevent double counting, certain conditions are typically applied:

Click Logging Flow

When a user clicks a short link, two things happen simultaneously:

To avoid slowing down the redirect path, these actions are separated into synchronous and asynchronous operations.

This decoupled design ensures the redirect latency remains low while analytics are processed asynchronously.

Design Approaches

Approach 1: Direct Increment (Simple, Suitable for Low Scale)

Each redirect increments a counter directly in the database or cache.

Pros

Cons

Approach 2: Buffered Counting (Recommended for Scale)

For high-traffic systems, direct increments are inefficient. Instead, accumulate clicks in memory or a fast cache and flush them periodically to the database.

Workflow

1. When a user clicks a link, increment a counter in Redis:

2. A background job periodically aggregates and updates the persistent store:

3. Reset the Redis counter after flushing.

Pros

Cons

Approach 3: Event Streaming (For Real-Time Analytics)

Large-scale systems such as Bitly or Google Analytics use event-driven pipelines.

Each click generates an event that flows through systems like:

This enables real-time dashboards and flexible queries across billions of clicks.

Pros

Cons

Quiz