{"title":"Design WhatsApp","description":null,"content":"\n> **QUESTION**\n>\n> #### What is WhatsApp?\n>\n> WhatsApp is a widely used instant messaging application that enables real-time communication between users through instant message delivery. Users can send text messages, media files, and other content to individuals or groups, with messages delivered within milliseconds.\n>\n> \n> \n>\n> The core idea is deceptively simple: User A sends a message, and User B receives it instantly. However, achieving this at scale with billions of users, while ensuring message delivery guarantees, handling offline users, and supporting group conversations, introduces significant distributed systems challenges.\n>\n> **Other Popular Examples:** Facebook Messenger, Telegram, Signal, WeChat\n\n\nIn this chapter, we will dive into the **high-level design of a messaging system like WhatsApp**.\n\nThis problem is a favorite in system design interviews because it touches on so many fundamental concepts: **real-time communication**, **persistent connections**, **message ordering**, **delivery guarantees**, and the challenges of building a truly **global-scale system**. \n\nLet's start by understanding what exactly we are building.\n\n---\n\n# 1. Clarifying Requirements\n\nBefore diving into the design, it's important to ask thoughtful questions to uncover hidden assumptions, clarify ambiguities, and define the system's scope more precisely.\n\nHere is an example of how a discussion between the candidate and the interviewer might unfold:\n\n\n> **DISCUSSION**\n>\n> **Candidate:** \"What is the expected scale? How many users and messages per day should the system support?\"\n>\n> **Interviewer:** \"Let's design for 500 million daily active users (DAU) sending an average of 40 messages per day.\"\n>\n> **Candidate:** \"Should we support only one-on-one messaging, or also group chats?\"\n>\n> **Interviewer:** \"Both. Group chats should support up to 500 members.\"\n>\n> **Candidate:** \"What types of content should messages support? Text only, or also media like images and videos?\"\n>\n> **Interviewer:** \"Focus on text messages for the core design. You can mention media handling at a high level, but detailed media processing is out of scope.\"\n>\n> **Candidate:** \"Do we need to show online/offline status and typing indicators?\"\n>\n> **Interviewer:** \"Yes, presence indicators (online/offline/last seen) are important. Typing indicators are nice-to-have.\"\n>\n> **Candidate:** \"What about message delivery guarantees? Should users see read receipts?\"\n>\n> **Interviewer:** \"Yes. Users should see when their message is delivered and when it's read. Messages should never be lost.\"\n>\n> **Candidate:** \"Should messages be stored permanently, or can they expire?\"\n>\n> **Interviewer:** \"Messages should be stored until explicitly deleted by the user. We need to support message history sync across devices.\"\n>\n> **Candidate:** \"What about end-to-end encryption?\"\n>\n> **Interviewer:** \"You can mention it conceptually, but detailed cryptographic implementation is out of scope.\"\n\n\nAfter gathering the details, we can summarize the key system requirements.\n\n## 1.1 Functional Requirements\n\n- **One-on-One Chat:** Users can send and receive messages in real-time with other users.\n- **Group Chat:** Users can create groups and send messages to multiple recipients (up to 500 members).\n- **Message Delivery Status:** Users can see delivery receipts (sent, delivered, read).\n- **Online Presence:** Users can see if their contacts are online, offline, or their last seen time.\n- **Message History:** Users can access their message history and sync across multiple devices.\n- **Push Notifications:** Offline users receive push notifications for new messages.\n\n\n> **Out of Scope**\n>\n> To keep our discussion focused, we will set aside a few features that, while important, would take us down rabbit holes:\n>\n> - **Media Messages:** Images, videos, voice notes (mentioned conceptually only).\n> - **Voice/Video Calls:** Real-time audio and video communication.\n> - **End-to-End Encryption:** Detailed cryptographic implementation.\n> - **Stories/Status Updates:** Ephemeral content sharing.\n\n\n## 1.2 Non-Functional Requirements\n\n- **Low Latency:** Messages should be delivered within milliseconds for online users. Target: p99 < 100ms for message delivery.\n- **High Availability:** The system must be highly available (99.99% uptime). Users expect messaging to work 24/7.\n- **Reliability:** Messages must never be lost. Once sent, a message should eventually be delivered, even if the recipient is offline.\n- **Scalability:** Support 500M+ daily active users and 20B+ messages per day.\n- **Ordering:** Messages within a conversation should appear in the correct order.\n- **Consistency:** Eventually consistent for presence, strong consistency for message delivery.\n\n---\n\n# 2. Back-of-the-Envelope Estimation\n\nWith our requirements clear, lets understand the scale we are dealing with. In most interviews, you are not required to do a detailed estimation.\n\n\n> **Assumptions**\n>\n> We will use these baseline numbers throughout our calculations:\n>\n> - **Daily Active Users (DAU):** 500 million\n> - **Messages per user per day:** 40\n> - **Average message size:** 100 bytes (text content + metadata)\n> - **Average group size:** 20 members\n> - **Percentage of group messages:** 30%\n\n\n#### Message Throughput\n\nLet's start with the fundamental question: how many messages flow through this system?\n\n- **Total messages per day:** 500 million users x 40 messages = **20 billion messages/day**\n\nTwenty billion. That is 20,000,000,000 messages every single day. Let's convert that to something more tangible:\n\n- **Average messages per second:** 20 billion / 86,400 seconds = **~230,000 messages/second**\n- **Peak load (3x average):** **~700,000 messages/second**\n\nThe 3x multiplier accounts for peak hours when everyone is awake and chatting. Traffic is never uniform throughout the day.\n\nThese numbers tell us something important: we are looking at hundreds of thousands of concurrent operations per second. This is not a system where we can make a database query for every message. We need persistent connections, efficient routing, and aggressive caching.\n\n#### Connection Load\n\nHere is where messaging systems get interesting, and fundamentally different from typical web applications. Unlike a website where users make requests and disconnect, a messaging app needs to push messages to users the instant they arrive. \n\nThat means maintaining persistent connections with every online user.\n\n- **Concurrent connections:** If 10% of DAU are online at any time = **50 million concurrent connections**\n- **Peak concurrent connections:** **~100 million**\n\nEach of these 50 million connections requires maintaining a persistent WebSocket. This is a fundamentally different challenge from handling 50 million HTTP requests per day. These connections stay open, consuming memory and file descriptors on our servers.\n\nIf a single well-tuned server can handle 50,000 concurrent WebSocket connections (a reasonable estimate for modern hardware with proper kernel tuning), we need:\n\n\n```shell\n50 million connections / 50,000 per server = 1,000 chat servers\n```\n\n\nJust for connection handling alone, we need a fleet of a thousand servers.\n\n#### Storage (Per Day)\n\nStorage requirements for a text-only system are more modest than you might expect:\n\n- **Message storage:** 20 billion messages x 100 bytes = **2 TB/day**\n- **Annual storage:** 2 TB x 365 = **730 TB/year** (just for messages)\n\nSeven hundred terabytes per year sounds substantial, but it is well within reach of modern distributed databases like Cassandra or ScyllaDB. For context, a single NVMe drive can hold 4 TB, so we are talking about a few hundred drives worth of storage.\n\nThe real challenge with storage is not capacity, it is the access patterns. We need to write 230,000 messages per second while simultaneously reading message history and syncing devices. Latency matters more than raw throughput.\n\n#### Bandwidth\n\nBandwidth is rarely the bottleneck for text messaging, but let's verify:\n\n- **Incoming bandwidth:** 230K msg/sec x 100 bytes = **~23 MB/sec (inbound)**\n- **Outgoing bandwidth:** Higher due to group message fanout\n\nWhen a message goes to a group of 20 members, it needs to reach 20 devices. If 30% of messages are group messages, outbound traffic multiplies accordingly.\n\nBut even accounting for this, we are looking at hundreds of megabytes per second, easily handled by modern network infrastructure.\n\n---\n\n# 3. Core APIs\n\nBefore diving into architecture, it helps to think about the API contract. What operations does our system need to support? \n\nDefining the APIs early forces us to think concretely about what users can do and what data flows through the system.\n\nA messaging system's API is unusual compared to typical web services. Most real-time communication happens over persistent WebSocket connections, not traditional HTTP request-response. \n\nHowever, we still need REST endpoints for operations that do not require instant delivery, like fetching message history.\n\n\n```mermaid\nflowchart LR\n subgraph \"Real-time (WebSocket)\"\n W1[Send Message]:::primary\n W2[Receive Message]:::primary\n W3[Typing Indicator]:::primary\n end\n\n subgraph \"Request/Response (REST)\"\n R1[Fetch History]:::orange\n R2[Update Status]:::orange\n R3[Get Presence]:::orange\n end\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n```\n\n\nLet's walk through the essential APIs.\n\n### **1. Send Message**\n\n#### Endpoint: `WebSocket message or POST /messages`\n\nThis is the heart of our system. When a user taps send, this API handles getting the message from their device to ours. \n\nIn practice, this almost always goes over the WebSocket connection for lowest latency, but having a REST fallback is useful when WebSocket connections fail.\n\n##### **Request Parameters:**\n\n\n| Parameter | Required | Description |\n|-----------|----------|-------------|\n| `sender_id` | Yes | ID of the user sending the message |\n| `recipient_id` | Yes | ID of the recipient user or group |\n| `message_type` | Yes | Whether the recipient is a `user` or `group` |\n| `content` | Yes | The actual message text |\n| `client_message_id` | Yes | Client-generated unique ID for deduplication |\n| `timestamp` | Yes | Client-side timestamp when the message was composed |\n\n\n##### **Sample Response:**\n\n\n```json\n{\n \"message_id\": \"msg_123abc\",\n \"status\": \"sent\",\n \"server_timestamp\": \"2024-01-15T10:30:00.123Z\"\n}\n```\n\n\nThe `client_message_id` deserves special attention. Networks are unreliable. A user might tap send, their phone loses connectivity for a moment, and the app retries the send. \n\nWithout deduplication, the recipient would see the same message twice. By including a client-generated ID, the server can detect and ignore duplicates, ensuring exactly-once delivery semantics.\n\n##### **Error Cases:**\n\n\n| Status Code | Meaning | When It Happens |\n|-------------|---------|-----------------|\n| `400 Bad Request` | Invalid input | Message too long, missing required fields |\n| `403 Forbidden` | Not authorized | Trying to send to a blocked user or private group |\n| `429 Too Many Requests` | Rate limited | Sending too many messages too quickly |\n\n\n### **2. Fetch Messages**\n\n#### Endpoint: `GET /conversations/{conversation_id}/messages`\n\nWhen a user opens an old conversation or logs in on a new device, they need to see their message history. This endpoint retrieves messages for a conversation, typically the most recent ones first.\n\n##### **Request Parameters:**\n\n\n| Parameter | Required | Description |\n|-----------|----------|-------------|\n| `conversation_id` | Yes | ID of the conversation to fetch |\n| `cursor` | No | Pagination cursor for fetching older messages |\n| `limit` | No | Number of messages to return (default: 50, max: 100) |\n\n\n##### **Sample Response:**\n\n\n```json\n{\n \"messages\": [\n {\n \"message_id\": \"msg_123\",\n \"sender_id\": \"user_456\",\n \"content\": \"Hey, how are you?\",\n \"timestamp\": \"2024-01-15T10:30:00Z\",\n \"status\": \"read\"\n }\n ],\n \"next_cursor\": \"eyJtc2dfaWQiOiJtc2dfMTIyIn0=\",\n \"has_more\": true\n}\n```\n\n\nNotice that we use cursor-based pagination rather than offset-based. With billions of messages, a query like `OFFSET 1000000` would be painfully slow, requiring the database to skip over a million rows. Cursor-based pagination uses an indexed value (like a message ID or timestamp) to efficiently jump to the right position.\n\n### **3. Update Message Status**\n\n#### Endpoint: `POST /messages/{message_id}/status`\n\nThis is what powers those checkmarks. When a message is delivered to the recipient's device or opened by the user, we need to update its status and notify the sender.\n\n##### **Request Parameters:**\n\n\n| Parameter | Required | Description |\n|-----------|----------|-------------|\n| `message_id` | Yes | ID of the message to update |\n| `status` | Yes | New status: `delivered` or `read` |\n| `timestamp` | Yes | When the status change occurred |\n\n\nStatus updates flow in one direction: `sent → delivered → read`. We never go backwards. The timestamp helps with edge cases where status updates arrive out of order due to network delays.\n\n### **4. Get User Presence**\n\n#### Endpoint: `GET /users/{user_id}/presence`\n\nReturns whether a user is currently online and, if offline, when they were last active. This powers the \"online\" indicator and \"last seen\" text in the UI.\n\n##### **Sample Response:**\n\n\n```json\n{\n \"user_id\": \"user_456\",\n \"status\": \"offline\",\n \"last_seen\": \"2024-01-15T09:45:00Z\"\n}\n```\n\n\nPresence is intentionally kept simple. We do not need to know exactly what a user is doing, just whether they are actively connected. Privacy controls allow users to hide their last seen time, in which case we simply omit that field. \n\nWith our API contract defined, we have a clear picture of what the system needs to do. Now let's design the architecture that makes these APIs work at scale.\n\n---\n\n# 4. High-Level Design\n\nNow we get to the heart of the design. Rather than throwing a complex architecture diagram at you with 15 boxes and wondering what each one does, we are going to build this system incrementally. \n\nWe will start with the simplest possible design that solves our first requirement, then add components only as we encounter new challenges. This mirrors how you should think through the problem in an interview.\n\nOur system must ultimately satisfy three core requirements:\n\n1. **Real-time Message Delivery:** Messages should reach online recipients within milliseconds.\n2. **Offline Message Handling:** Messages for offline users need to be stored and delivered when they come back online.\n3. **Group Message Distribution:** A single message should efficiently reach all group members.\n\nBefore we dive into the architecture, let's understand the key insight that shapes everything: messaging is fundamentally a **push-based** system.\n\nThink about how a typical web application works. Your browser requests a page, the server responds, and the connection closes. If you want new data, you request again. This request-response pattern works great for most applications, but it falls apart for messaging. \n\nYou cannot expect users to constantly refresh to check for new messages. The moment a message arrives at our servers, we need to push it to the recipient's device immediately.\n\n\n```mermaid\nflowchart TD\n\n subgraph Messaging[\"Push-based\"]\n C2[Client]:::primary\n S2[Server]:::orange\n C2 <-->|\"Persistent Connection\"| S2\n S2 -->|\"Push new message\"| C2\n end\n\n subgraph Traditional[\"Pull-based\"]\n C1[Client]:::primary\n S1[Server]:::orange\n C1 -->|\"Request\"| S1\n S1 -->|\"Response\"| C1\n end\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n```\n\n\nThis push-based nature is why we need persistent WebSocket connections rather than traditional HTTP. And maintaining millions of persistent connections creates a whole set of challenges that we need to address.\n\nLet's start building, one requirement at a time.\n\n---\n\n## 4.1 Requirement 1: Real-time One-on-One Messaging\n\nLet's start with the simplest possible scenario: User A sends a message to User B, and User B is currently online with the app open. \n\n**What do we need to make this work?**\n\nThe naive approach might be: store the message in a database and have User B periodically check for new messages. But polling introduces latency and wastes resources. \n\nWe need to push the message the instant it arrives. This means maintaining a persistent connection between our servers and User B's device.\n\n### The Components We Need\n\nLet's introduce the components one by one, understanding why each exists.\n\n#### **Chat Servers**\n\nThese are the workhorses of our system. Each chat server maintains persistent WebSocket connections with thousands of clients simultaneously.\n\nWhen User A opens the messaging app, their phone establishes a WebSocket connection to one of the chat servers. This connection stays open for as long as the app is in use. When User A sends a message, it travels over this existing connection, no need to establish a new one.\n\n##### **What chat servers do:**\n\n- Accept and maintain WebSocket connections from clients\n- Receive messages from senders\n- Route messages to the right recipients (which might be on different chat servers)\n- Send heartbeats to detect dead connections\n- Handle reconnection when users switch networks\n\nHere is an important insight: chat servers are **stateful**. Unlike typical web servers where any server can handle any request, User B's messages must go to the specific chat server where User B's connection lives. If User B is connected to Chat Server 2, sending their message to Chat Server 1 will not help.\n\n\n```mermaid\nflowchart TB\n\n\tsubgraph CS3[\"Chat Server 3\"]\n U6[\"User F\"]:::primary\n U7[\"User G\"]:::primary\n end\n\n subgraph CS2[\"Chat Server 2\"]\n U4[\"User B\"]:::orange\n U5[\"User E\"]:::primary\n end\n\n subgraph CS1[\"Chat Server 1\"]\n U1[\"User A\"]:::primary\n U2[\"User C\"]:::primary\n U3[\"User D\"]:::primary\n end\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n```\n\n\nThis statefulness creates a routing challenge. When User A sends a message to User B, how does Chat Server 1 know that User B is on Chat Server 2?\n\n#### **Session Service**\n\nThis is where the Session Service comes in. It maintains a simple but critical mapping: which user is connected to which chat server.\n\nWhen User B connects to Chat Server 2, that server registers the connection: \"User B is on Chat Server 2.\" When User A wants to send a message to User B, we query the Session Service: \"Where is User B?\" It responds: \"Chat Server 2.\"\n\n##### **What the Session Service does:**\n\n- Track which chat server each online user is connected to\n- Update mappings in real-time as users connect and disconnect\n- Provide sub-millisecond lookups for message routing\n\nWe typically implement this using **Redis** because it offers exactly what we need: fast key-value lookups with built-in expiration for handling disconnections. The data structure is simple:\n\n\n```shell\nuser_a -> chat_server_1\nuser_b -> chat_server_2\nuser_c -> chat_server_1\n...\n```\n\n\n#### **Message Service**\n\nWhile routing messages in real-time is essential, we also need to persist them. Users expect to see their message history. If User B's phone dies right as a message arrives, we do not want to lose it.\n\n##### **What the Message Service does:**\n\n- Persist every message to the database before attempting delivery\n- Generate server-side message IDs and timestamps (for ordering)\n- Track message status (sent, delivered, read)\n- Provide message history queries for the Fetch Messages API\n\n### Putting It Together: The Message Flow\n\nNow let's trace what happens when User A sends \"Hey, how's it going?\" to User B. Both users are online, connected to different chat servers.\n\n\n```mermaid\nsequenceDiagram\n participant A as User A\n participant CS1 as Chat Server 1\n participant SS as Session Service\n participant MS as Message Service\n participant DB as Database\n participant CS2 as Chat Server 2\n participant B as User B\n\n A->>CS1: 1. Send message to User B\n CS1->>MS: 2. Persist message\n MS->>DB: Store message\n DB-->>MS: Confirm\n MS-->>CS1: 3. Message ID + timestamp\n CS1->>SS: 4. Where is User B?\n SS-->>CS1: 5. Chat Server 2\n CS1->>CS2: 6. Forward message\n CS2->>B: 7. Push via WebSocket\n B-->>CS2: 8. ACK (delivered)\n CS2->>MS: 9. Update status\n CS2->>CS1: 10. Delivery confirmation\n CS1->>A: 11. Message delivered ✓✓\n```\n\n\nLet's walk through each step to understand what is happening:\n\n**Step 1-3: Receive and persist**\n\nUser A taps send. The message travels over the existing WebSocket connection to Chat Server 1. Before doing anything else, Chat Server 1 asks the Message Service to persist the message. This is critical. If we route the message first and something fails, the message could be lost. By persisting first, we guarantee that no matter what happens next, the message is safely stored.\n\nThe Message Service writes the message to the database and returns a server-generated message ID and timestamp. The timestamp is important because the server's clock is the source of truth for message ordering, not the client's clock which might be wrong.\n\n**Step 4-5: Find the recipient**\n\nWith the message safely stored, Chat Server 1 needs to find User B. It queries the Session Service: \"Where is User B connected?\" The Session Service responds: \"Chat Server 2.\" This lookup takes less than a millisecond thanks to Redis.\n\n**Step 6-7: Route and deliver**\n\nChat Server 1 forwards the message to Chat Server 2. This happens over a direct connection between servers, typically using gRPC or a similar efficient protocol. Chat Server 2 receives the message and pushes it to User B over their WebSocket connection.\n\n**Step 8-11: Acknowledge and confirm**\n\nUser B's client receives the message and sends an acknowledgment back. This ACK travels back through the system, updating the message status to \"delivered\" in the database along the way. Finally, User A's client receives the delivery confirmation and updates the UI to show the double checkmark.\n\nThis entire round trip, from User A tapping send to seeing the delivered checkmark, typically completes in under 100 milliseconds when both users are online. That is fast enough that conversations feel instantaneous.\n\nBut here is the question that should be nagging at you: **what happens when User B is not online?**\n\n---\n\n## 4.2 Requirement 2: Handling Offline Users\n\nThe flow we just designed works beautifully when both users are online. But real-world messaging is messier. What happens when User B's phone is in airplane mode? What if they have not opened the app in hours? What if they are in a subway tunnel with no signal?\n\nWe cannot just drop the message. This would violate our reliability requirement. Users expect that once they tap send, the message will eventually arrive, even if the recipient is unreachable for hours or days.\n\nThis requirement forces us to think differently about message delivery. We cannot just push a message and forget about it. We need to track pending deliveries and retry when users come back online.\n\n### New Components for Offline Handling\n\nLet's introduce two new pieces to our architecture.\n\n#### **Message Queue**\n\nThink of the message queue as a mailbox. When we discover that User B is offline, instead of dropping the message, we place it in User B's queue. The messages sit there, safe and ordered, until User B comes back online.\n\n##### **What the message queue does:**\n\n- Store messages for offline users in order\n- Track which messages have been delivered and which are pending\n- Provide fast retrieval when users reconnect\n- Handle the case where users are offline for extended periods\n\nWe typically use a system like **Kafka** or **Redis Streams** for this. The key insight is that this is not the same as our message database. The database is for long-term storage and history. The queue is for pending deliveries, messages that have been persisted but not yet delivered to the recipient's device.\n\n#### **Push Notification Service**\n\nEven though we cannot deliver the message content directly to an offline user, we can still tell them something is waiting. This is where push notifications come in.\n\n##### **What the Push Notification Service does:**\n\n- Integrate with Apple Push Notification Service (APNs) for iOS\n- Integrate with Firebase Cloud Messaging (FCM) for Android\n- Send a notification like \"New message from User A\" to wake up the device\n- Respect user preferences (muted conversations, quiet hours)\n\n### The Offline Message Flow\n\nLet's trace what happens when User A sends a message but User B is offline.\n\n#### **When the message is sent:**\n\n\n```mermaid\nflowchart TD\n A[User A sends message]:::primary --> CS1[Chat Server 1]\n CS1 --> MS[Message Service]\n MS --> DB[(Database)]\n\n CS1 --> SS{Session Service:
Is User B online?}\n\n SS -->|Online| ROUTE[Route to Chat Server]\n SS -->|Offline| MQ[Message Queue]\n\n MQ --> PNS[Push Notification
Service]\n PNS --> APNS[APNs / FCM]\n APNS --> Device[User B's Device]\n\n style A fill:#00ceff,stroke:#000,color:#000\n style CS1 fill:#38d9a9,stroke:#000,color:#000\n style MS fill:#38d9a9,stroke:#000,color:#000\n style DB fill:#9775fa,stroke:#000,color:#000\n style SS fill:#ffa94d,stroke:#000,color:#000\n style MQ fill:#69db7c,stroke:#000,color:#000\n style PNS fill:#da77f2,stroke:#000,color:#000\n style APNS fill:#f783ac,stroke:#000,color:#000\n style Device fill:#f783ac,stroke:#000,color:#000\n```\n\n\n1. User A sends a message to User B. The message arrives at Chat Server 1.\n2. Chat Server 1 persists the message via the Message Service. It is now safely stored.\n3. Chat Server 1 queries the Session Service: \"Where is User B?\"\n4. The Session Service finds no entry for User B, meaning they are offline.\n5. Instead of failing, Chat Server 1 adds the message to User B's message queue.\n6. The Push Notification Service sends a notification to User B's phone: \"New message from User A.\"\n7. User A sees a single checkmark (sent) but not a double checkmark (delivered) yet.\n\n#### **When User B comes back online:**\n\n\n```mermaid\nflowchart TD\n subgraph Later[\"When User B Comes Online\"]\n B_CONNECT[User B connects]:::green --> CS3[Chat Server]\n CS3 --> FETCH[Fetch queued messages]\n FETCH --> MQ2[(Message Queue)]\n MQ2 --> DELIVER[Deliver messages]\n end\n\n style B_CONNECT fill:#69db7c,stroke:#000,color:#000\n```\n\n\n1. User B opens the app and establishes a WebSocket connection to a chat server (maybe Chat Server 3 this time).\n2. During the connection handshake, Chat Server 3 checks the message queue: \"Are there any pending messages for User B?\"\n3. The queue returns all pending messages, ordered by timestamp.\n4. Chat Server 3 pushes these messages to User B over the WebSocket connection.\n5. User B's client acknowledges receipt.\n6. The queue entries are cleared, and the message statuses are updated to \"delivered.\"\n7. Back at User A's device, the checkmarks update from single to double.\n\nThe beauty of this design is that the message is never lost. Whether User B comes online in 10 seconds or 10 days, the message will be waiting. The queue acts as a reliable buffer between the sender and an unreachable recipient.\n\n\n> **TIP**\n>\n> We persist the message to the database before adding it to the queue. This means even if the queue itself fails (a rare event), we have not lost the message. The database is our source of truth; the queue is just an optimization for fast delivery.\n\n\n---\n\n## 4.3 Requirement 3: Group Messaging\n\nSo far we have handled one-on-one messaging elegantly. But groups introduce a new challenge that fundamentally changes our design: **fanout**.\n\nConsider this scenario: User A sends \"Happy New Year!\" to a family group with 50 members. That single message needs to reach 50 different devices, potentially scattered across 20 different chat servers, some members online and some offline, some on fast WiFi and some on spotty mobile networks.\n\nWith one-on-one messaging, one input means one output. With groups, one input means many outputs. This multiplier effect is called fanout, and it can easily overwhelm a naive implementation.\n\n### Understanding the Fanout Problem\n\nLet's visualize what happens when a message goes to a group:\n\n\n```mermaid\nflowchart TD\n MSG[\"User A sends:
'Hello everyone!'\"]:::primary\n\n MSG --> FANOUT{Fanout Required}\n\n FANOUT --> M1[Member 1
Chat Server 1]:::secondary\n FANOUT --> M2[Member 2
Chat Server 1]:::secondary\n FANOUT --> M3[Member 3
Chat Server 2]:::secondary\n FANOUT --> M4[Member 4
Chat Server 2]:::secondary\n FANOUT --> M5[Member 5
Chat Server 3]:::secondary\n FANOUT --> M6[\"... Member 100
Chat Server N\"]:::secondary\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef secondary fill:#38d9a9,stroke:#000,color:#000\n```\n\n\nIf User A sends a message to a group with 500 members (our maximum), and the sender's chat server has to individually deliver to all 500, we have a problem:\n\n- The sender's chat server becomes a bottleneck\n- If it crashes mid-delivery, some members get the message and some do not\n- Large groups would create noticeable delays\n\nThere are several ways to handle fanout. Let's examine each and understand their trade-offs.\n\n### **Approach 1: Sender-Side Fanout**\n\nThe simplest approach is to have the sender's chat server do all the work. When User A sends a group message, Chat Server 1 looks up all group members, finds their chat servers, and delivers to each one.\n\n\n```mermaid\nflowchart TD\n A[User A]:::primary --> CS1[Chat Server 1]\n CS1 --> M1[Member 1]:::secondary\n CS1 --> M2[Member 2]:::secondary\n CS1 --> M3[Member 3]:::secondary\n CS1 --> M4[...]:::secondary\n CS1 --> M100[Member 100]:::secondary\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef secondary fill:#38d9a9,stroke:#000,color:#000\n```\n\n\n**How it works:**\n\n1. Chat Server 1 queries the Group Service for all members of the group\n2. For each member, it queries the Session Service to find their chat server\n3. It forwards the message to each destination chat server\n4. Each chat server delivers to their connected members\n\n**The good:**\n\n- Simple to implement and reason about\n- Low latency for small groups since there is no intermediary\n- No additional infrastructure required\n\n**The bad:**\n\n- A single server becomes the bottleneck for large groups\n- If that server crashes mid-fanout, delivery is incomplete\n- A 500-member group could take several seconds to process\n\nThis approach works fine for small groups (under 50-100 members), which are the majority of groups in typical usage patterns.\n\n### **Approach 2: Message Queue Fanout**\n\nFor larger groups, we can use a message queue to distribute the work across multiple workers.\n\n\n```mermaid\nflowchart LR\n A[User A]:::primary --> CS1[Chat Server 1]\n CS1 --> K[Kafka Topic:
group_123]:::purple\n\n K --> W1[Worker 1]:::orange\n K --> W2[Worker 2]:::orange\n K --> W3[Worker 3]:::orange\n\n W1 --> M1[Members 1-33]:::secondary\n W2 --> M2[Members 34-66]:::secondary\n W3 --> M3[Members 67-100]:::secondary\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef secondary fill:#38d9a9,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n```\n\n\n**How it works:**\n\n1. Chat Server 1 publishes the message to a Kafka topic for the group\n2. Multiple worker processes consume from this topic in parallel\n3. Each worker is responsible for delivering to a subset of group members\n4. The work is automatically distributed based on worker capacity\n\n**The good:**\n\n- Scales horizontally by adding more workers\n- Resilient to individual worker failures (Kafka retries automatically)\n- No single point of bottleneck\n\n**The bad:**\n\n- Adds latency since messages pass through the queue\n- More complex infrastructure to manage\n- Overkill for small groups where direct delivery is faster\n\n### **Approach 3: Hybrid Approach (Recommended)**\n\nThe smart solution is to combine both approaches, choosing based on group size:\n\n\n```mermaid\nflowchart TB\n MSG[Group Message]:::primary\n\n MSG --> CHECK{Group Size?}:::orange\n\n CHECK -->|\"< 100 members\"| DIRECT[\"Direct Fanout
from Sender's Server\"]:::green\n CHECK -->|\">= 100 members\"| QUEUE[\"Kafka Queue
Distributed Fanout\"]:::purple\n\n DIRECT --> D1[Member 1]:::secondary\n DIRECT --> D2[Member 2]:::secondary\n DIRECT --> D3[...]:::secondary\n\n QUEUE --> W1[\"Worker 1
Members 1-50\"]:::secondary\n QUEUE --> W2[\"Worker 2
Members 51-100\"]:::secondary\n QUEUE --> W3[\"Worker N
Members...\"]:::secondary\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef secondary fill:#38d9a9,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n```\n\n\n- **Small groups (under 100 members):** Use direct fanout from the sender's server. This is faster and simpler, and most groups fall into this category.\n- **Large groups (100+ members):** Route through Kafka for distributed fanout. The added latency is acceptable for groups where reliability and scalability matter more.\n\nThe threshold of 100 is not magic; it is a tunable parameter based on your server capacity. The key insight is that different group sizes warrant different delivery strategies. This hybrid approach gives us the best of both worlds: low latency for the common case and scalability for the edge cases.\n\n### The Complete Group Message Flow\n\nLet's put it all together and trace a group message from send to delivery:\n\n\n```mermaid\nflowchart TD\n A[User A sends
group message]:::primary --> CS1[Chat Server 1]\n CS1 --> MS[Message Service]\n CS1 --> GS[Group Service:
Get members]\n GS --> CS1\n CS1 --> SS[Session Service:
Get member locations]\n SS --> CS1\n\n CS1 --> CS2[Chat Server 2]\n CS1 --> CS3[Chat Server 3]\n CS1 --> MQ[Message Queue
for offline users]\n\n CS2 --> B[User B]:::rose\n CS2 --> C[User C]:::rose\n CS3 --> D[User D]:::rose\n\n style A fill:#00ceff,stroke:#000,color:#000\n style CS1 fill:#38d9a9,stroke:#000,color:#000\n style MS fill:#38d9a9,stroke:#000,color:#000\n style GS fill:#38d9a9,stroke:#000,color:#000\n style SS fill:#ffa94d,stroke:#000,color:#000\n style CS2 fill:#38d9a9,stroke:#000,color:#000\n style CS3 fill:#38d9a9,stroke:#000,color:#000\n style MQ fill:#69db7c,stroke:#000,color:#000\n style B fill:#f783ac,stroke:#000,color:#000\n style C fill:#f783ac,stroke:#000,color:#000\n style D fill:#f783ac,stroke:#000,color:#000\n```\n\n\n#### **Step by step:**\n\n1. **Persist first:** User A sends a message to the group. Chat Server 1 immediately persists it via the Message Service with the `group_id` instead of a single recipient. The message is now durable.\n2. **Get members:** Chat Server 1 queries the Group Service: \"Who are the members of this group?\" The Group Service returns the list of member IDs.\n3. **Find member locations:** For each member, we need to know where they are connected (or if they are offline). Instead of making 50 individual queries, we batch this: \"Where are users [B, C, D, E, ...]?\" The Session Service returns a map of user IDs to chat server locations.\n4. **Smart batching:** Here is a key optimization. Instead of forwarding to each member individually, we group members by their chat server. If members B, C, and E are all on Chat Server 2, we send one message to Chat Server 2 with all three recipient IDs. This dramatically reduces network overhead.\n5. **Parallel delivery:** Chat Server 1 sends messages to Chat Server 2 and Chat Server 3 in parallel. Each receiving chat server pushes the message to its connected members.\n6. **Handle offline members:** For members who are offline, the message goes to their queue. When they reconnect, they will receive it along with any other pending messages.\n\nThis flow handles groups of any size efficiently. For small groups, it completes in tens of milliseconds. For larger groups using the queue-based approach, delivery might take a bit longer but remains reliable.\n\n---\n\n## 4.4 Putting It All Together\n\nWe have now addressed each requirement incrementally. Let's step back and see the complete picture. This is the architecture you would draw on the whiteboard after explaining each component:\n\n\n```mermaid\nflowchart TB\n subgraph Clients\n C1[Mobile App]\n C2[Web App]\n end\n\n subgraph EdgeLayer[Edge Layer]\n LB[Load Balancer]\n APIGW[API Gateway]\n end\n\n subgraph ChatLayer[Chat Layer]\n CS1[Chat Server 1]\n CS2[Chat Server 2]\n CS3[Chat Server N]\n end\n\n subgraph Services\n MS[Message Service]\n GS[Group Service]\n US[User Service]\n PNS[Push Notification
Service]\n end\n\n subgraph DataLayer[Data Layer]\n Redis[(Redis
Session Store)]\n Kafka[Kafka
Message Queue]\n DB[(Message DB
Cassandra)]\n UserDB[(User DB
PostgreSQL)]\n end\n\n C1 & C2 <-->|WebSocket| LB\n C1 & C2 -->|REST API| APIGW\n LB <--> CS1 & CS2 & CS3\n APIGW --> MS & GS & US\n\n CS1 & CS2 & CS3 <--> Redis\n CS1 & CS2 & CS3 --> MS\n CS1 & CS2 & CS3 --> Kafka\n\n MS --> DB\n GS --> UserDB\n US --> UserDB\n Kafka --> PNS\n\n style C1 fill:#00ceff,stroke:#000,color:#000\n style C2 fill:#00ceff,stroke:#000,color:#000\n style LB fill:#38d9a9,stroke:#000,color:#000\n style APIGW fill:#38d9a9,stroke:#000,color:#000\n style CS1 fill:#ffa94d,stroke:#000,color:#000\n style CS2 fill:#ffa94d,stroke:#000,color:#000\n style CS3 fill:#ffa94d,stroke:#000,color:#000\n style MS fill:#69db7c,stroke:#000,color:#000\n style GS fill:#69db7c,stroke:#000,color:#000\n style US fill:#69db7c,stroke:#000,color:#000\n style PNS fill:#da77f2,stroke:#000,color:#000\n style Redis fill:#ff8787,stroke:#000,color:#000\n style Kafka fill:#9775fa,stroke:#000,color:#000\n style DB fill:#9775fa,stroke:#000,color:#000\n style UserDB fill:#9775fa,stroke:#000,color:#000\n```\n\n\nLooking at this architecture, we can identify distinct layers, each with a specific responsibility:\n\n**Client Layer:** Mobile apps and web browsers connect to our system. From our perspective, they are all just WebSocket clients sending and receiving JSON messages.\n\n**Edge Layer:** The load balancer distributes incoming connections across chat servers. For WebSocket connections, we typically use sticky sessions (or consistent hashing by user ID) so that a user's connection stays on the same server after initial assignment.\n\n**Real-time Chat Layer:** The fleet of chat servers handles all the persistent connections. These are stateful servers, meaning they remember which users are connected to them. This is fundamentally different from stateless web servers where any server can handle any request.\n\n**Service Layer:** These are traditional stateless services handling specific domains: messages, groups, users, and presence. They can scale horizontally without coordination.\n\n**Data Layer:** Redis provides fast, ephemeral storage for session mappings and presence. Kafka queues messages for reliable delivery. Cassandra stores the actual message history, optimized for write-heavy, time-ordered data. PostgreSQL handles user and group data where we need transactions and complex queries.\n\n\n| Component | Purpose | Why This Technology? |\n|-----------|---------|---------------------|\n| **Load Balancer** | Distributes WebSocket connections across chat servers | Sticky sessions for connection persistence |\n| **Chat Servers** | Maintain persistent connections, route messages in real-time | Stateful, handles 50K+ connections each |\n| **API Gateway** | Handles REST API requests for non-real-time operations | Rate limiting, authentication |\n| **Session Service (Redis)** | Maps users to their connected chat server | Sub-millisecond lookups, pub/sub for presence |\n| **Message Service** | Handles message persistence and retrieval | Decouples chat servers from storage |\n| **Group Service** | Manages group membership and metadata | ACID transactions for consistency |\n| **Presence Service** | Tracks online/offline status | Real-time updates via Redis |\n| **Message Queue (Kafka)** | Buffers messages for offline users, handles fanout | Durability, ordering guarantees |\n| **Push Notification Service** | Sends push notifications via APNs/FCM | Async processing, retry logic |\n| **Message Database (Cassandra)** | Stores message history | Write-optimized, time-series friendly |\n| **User Database (PostgreSQL)** | Stores user profiles and relationships | Complex queries, transactions |\n\n\nWith the high-level architecture clear, let's dive into how we store all this data efficiently.\n\n---\n\n# 5. Database Design\n\nThe database layer can make or break a messaging system. With 20 billion messages per day and 500 million users, we need to make careful choices. The wrong database will become a bottleneck that is painful to fix later.\n\nLet's think through the requirements and choose appropriately.\n\n## 5.1 SQL vs NoSQL\n\nOne of the most common mistakes in system design is treating all data the same. A messaging system has two fundamentally different types of data, and each deserves a different storage strategy.\n\n### Message Data: Write-Heavy and Time-Ordered\n\nThink about how we access messages:\n\n- **Write-heavy workload:** We are writing 230,000 messages per second. Every single message needs to be persisted.\n- **Simple queries:** \"Get me the last 50 messages for this conversation.\" No complex joins, no aggregations.\n- **Time-series nature:** Recent messages are accessed constantly. Messages from last year are rarely touched.\n- **No transactions needed:** A message either exists or it does not. We do not need to atomically update multiple messages.\n- **High availability is critical:** If the message database goes down, messaging stops.\n\nGiven these patterns, a **wide-column NoSQL database** like **Apache Cassandra** or **ScyllaDB** is the right choice:\n\n- **Built for writes:** Cassandra's log-structured storage handles high write throughput beautifully\n- **Linear scalability:** Need more capacity? Add more nodes. Cassandra distributes data automatically.\n- **Time-series optimization:** Clustering keys keep messages in a conversation sorted by time on disk\n- **Tunable consistency:** We can trade between consistency and availability per query\n\n### User and Group Data: Relational and Consistent\n\nNow think about user and group data:\n\n- **Complex relationships:** Which groups does this user belong to? Who are the admins of this group?\n- **Transactions:** When adding a user to a group, we need to update the group membership and the user's group list atomically\n- **Strong consistency:** If I just added you to a group, you should see it immediately\n- **Read-heavy:** User profiles are read often but updated rarely\n\nFor this, a **relational database** like **PostgreSQL** makes more sense:\n\n- **Rich query capabilities:** Joins, subqueries, and complex filters\n- **ACID transactions:** Multi-table updates are atomic\n- **Strong consistency:** Reads always see the latest writes\n- **Mature ecosystem:** Well-understood operations, excellent tooling\n\n## 5.2 Database Schema\n\nWith our database choices made, let's design the actual schemas. We have three categories of data, each stored in the technology best suited for it:\n\n### **1. Messages Table (Cassandra)**\n\nThis is the heart of our storage layer. The schema design is driven by a single question: \"What is the most common query we need to answer?\"\n\nFor a messaging app, that query is: **\"Get the last 50 messages for this conversation, ordered by time.\"**\n\nWe design the entire table around this access pattern:\n\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `conversation_id` | UUID (Partition Key) | Unique identifier for the conversation |\n| `message_id` | TimeUUID (Clustering Key) | Time-based UUID for ordering |\n| `sender_id` | UUID | ID of the message sender |\n| `content` | Text | Message content |\n| `message_type` | Text | Type: `text`, `image`, `video` |\n| `status` | Text | Delivery status: `sent`, `delivered`, `read` |\n| `created_at` | Timestamp | Server timestamp |\n\n\nLet's understand why each field is where it is:\n\n**Partition Key (**`conversation_id`**):** This determines which nodes store the data. All messages in a single conversation live together on the same nodes. When we query \"last 50 messages for conversation X\", Cassandra knows exactly which nodes to ask. This is what makes reads fast.\n\n**Clustering Key (**`message_id`** as TimeUUID):** Within a partition (a conversation), messages are physically sorted on disk by the clustering key. A TimeUUID is a special UUID that encodes the timestamp, so messages are automatically ordered by time. Fetching \"the last 50 messages\" becomes a simple range scan, not a full table scan.\n\nThe combination of partition key and clustering key means that our most common query, \"get recent messages for a conversation\", hits a single partition on a small number of nodes and reads data that is already sorted. This is as fast as it gets.\n\n### **2. User Conversations Table (Cassandra)**\n\nWhen a user opens the app, the first thing they see is their conversation list. We need to answer: \"What are this user's recent conversations, and what was the last message in each?\"\n\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `user_id` | UUID (Partition Key) | User ID |\n| `conversation_id` | UUID (Clustering Key) | Conversation ID |\n| `last_message_at` | Timestamp | Time of last message |\n| `unread_count` | Integer | Number of unread messages |\n| `last_message_preview` | Text | Preview of last message |\n\n\nNotice that we store `last_message_preview` directly in this table. This is intentional denormalization. When rendering the conversation list, we can show \"Hey, are you coming for lunch...\" without querying the messages table at all. \n\nIn a normalized design, we would have to join or make a second query. Here, one query gives us everything we need.\n\nThis is a common pattern in Cassandra: store the data in the shape you need to read it, even if it means duplicating information across tables.\n\n### **3. Groups Table (PostgreSQL)**\n\nGroup metadata lives in PostgreSQL where we can use proper relational modeling:\n\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `group_id` | UUID (PK) | Unique group identifier |\n| `name` | VARCHAR(100) | Group name |\n| `creator_id` | UUID (FK) | User who created the group |\n| `created_at` | Timestamp | Creation time |\n| `member_count` | Integer | Number of members |\n\n\nThe `member_count` is denormalized here even though we could compute it from the members table. This avoids a COUNT query every time we need to display group info.\n\n### **4. Group Members Table (PostgreSQL)**\n\nThis is the join table that maps users to groups:\n\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `group_id` | UUID (PK, FK) | Group ID |\n| `user_id` | UUID (PK, FK) | User ID |\n| `role` | VARCHAR(20) | Role: `admin`, `member` |\n| `joined_at` | Timestamp | When user joined |\n\n\nThe composite primary key `(group_id, user_id)` serves two purposes:\n\n1. It ensures a user can only be in a group once (no duplicates)\n2. It creates an index that allows efficient queries in both directions: \"all members of group X\" and \"all groups user Y belongs to\"\n\nWith these tables, we can handle all group operations with standard SQL queries and proper transaction support. When a user joins a group, we update both the membership table and the group's member_count in a single transaction.\n\nNow let's move on to the most interesting part of the design: the deep dive into specific challenges.\n\n---\n\n# 6. Design Deep Dive\n\nThe high-level architecture gives us the skeleton, but interviewers often want to probe deeper into specific areas. This is where you demonstrate not just that you know what components to use, but that you understand how they work and why certain approaches are better than others.\n\nLet's explore the trickiest aspects of building a messaging system.\n\n## 6.1 WebSocket vs Long Polling vs Server-Sent Events\n\nWe have mentioned WebSocket connections throughout this design, but why WebSocket specifically? \n\nThere are several ways to achieve real-time communication between clients and servers. Each has different trade-offs in terms of latency, resource usage, and complexity. \n\nLet's understand them so you can explain the choice in an interview.\n\n### **Approach 1: HTTP Long Polling**\n\nLong polling is the oldest technique for achieving real-time-like behavior with plain HTTP. It predates WebSockets and was the backbone of early real-time web apps like Gmail's chat.\n\n**The idea is simple:** the client makes an HTTP request asking \"any new messages for me?\" Instead of responding immediately with \"no,\" the server holds the connection open. If a new message arrives while the connection is open, the server responds with it immediately. If nothing happens for 30-60 seconds, the server responds with an empty result, and the client immediately makes another request.\n\n\n```mermaid\nsequenceDiagram\n participant C as Client\n participant S as Server\n\n C->>S: HTTP Request (any new messages?)\n Note over S: Hold connection open
(up to 30-60 seconds)\n S-->>C: Response (here's a new message!)\n\n C->>S: HTTP Request (any new messages?)\n Note over S: Hold connection...
waiting for data...\n Note over S: Timeout reached\n S-->>C: Empty response (nothing new)\n\n C->>S: HTTP Request (any new messages?)\n Note over S: Message arrives!\n S-->>C: Response (new message!)\n```\n\n\nThe client creates a continuous loop of requests, effectively creating a \"persistent\" connection using standard HTTP semantics.\n\n**The good:**\n\n- Works everywhere, including through corporate proxies and strict firewalls that block other protocols\n- Uses standard HTTP infrastructure, so load balancers and caching work as expected\n- Simple fallback when more modern approaches are blocked\n\n**The bad:**\n\n- Each request cycle involves TCP connection setup, HTTP headers, and potential TLS handshakes, all overhead that adds up\n- There is inherent latency: a message might arrive just after the last response, requiring the user to wait for the next polling cycle\n- Servers hold many mostly-idle connections, wasting resources\n\nLong polling got us through the early web era, but it is not ideal for a modern messaging system with millions of concurrent users.\n\n### **Approach 2: Server-Sent Events (SSE)**\n\nSSE improves on long polling by establishing a true persistent connection, but only in one direction. The server can push events to the client continuously, but the client still needs to use regular HTTP requests to send data back.\n\n\n```mermaid\nsequenceDiagram\n participant C as Client\n participant S as Server\n\n C->>S: Open SSE connection\n Note over C,S: Persistent HTTP connection established\n\n S-->>C: Event: new message from Alice\n S-->>C: Event: Alice is typing...\n S-->>C: Event: new message from Alice\n\n Note over C: Need to send a message\n C->>S: POST /messages (separate HTTP request)\n\n S-->>C: Event: message delivered ✓\n```\n\n\nThink of SSE as a one-way pipe from server to client. The server can push events whenever it wants, but sending a message back requires a separate HTTP POST.\n\n**The good:**\n\n- Eliminates the request/response cycle of long polling\n- Built-in reconnection handling when connections drop\n- Works well with HTTP/2, which can multiplex multiple streams\n\n**The bad:**\n\n- Fundamentally unidirectional, so chat requires a hybrid approach: SSE for receiving, HTTP for sending\n- This asymmetry adds complexity and slightly higher latency for outgoing messages\n- Not as universally supported as WebSockets in mobile SDKs\n\nSSE is a good fit for notification streams, live feeds, or stock tickers where the server broadcasts and the client mostly listens. For chat, where both sides constantly send data, we need something better.\n\n### **Approach 3: WebSocket (Recommended)**\n\nWebSocket is the modern solution. It provides a true bidirectional channel where both client and server can send messages at any time, over a single persistent TCP connection.\n\n\n```mermaid\nsequenceDiagram\n participant C as Client\n participant S as Server\n\n C->>S: HTTP Upgrade: WebSocket\n S-->>C: 101 Switching Protocols\n Note over C,S: Full-duplex connection established\n\n C->>S: Send message to Bob\n S-->>C: Message sent ✓\n\n S-->>C: New message from Alice\n C->>S: Mark as read\n\n S-->>C: Alice is typing...\n C->>S: I'm typing too...\n\n Note over C,S: Both sides can send
at any time\n```\n\n\nThe connection starts with a standard HTTP request that includes an \"Upgrade\" header. If the server supports WebSocket, it responds with 101 Switching Protocols, and from that point on, the connection is a full WebSocket. Both sides can send frames whenever they want, there is no request/response dance.\n\n**The good:**\n\n- True bidirectional communication: either side can initiate a message at any time\n- Minimal overhead after connection: just the WebSocket frame header (as small as 2 bytes)\n- Lowest possible latency: messages are pushed instantly\n- Single connection for all traffic, reducing connection setup costs\n\n**The bad:**\n\n- Stateful nature complicates scaling: you need to track which server each user is on\n- Requires WebSocket-aware load balancers and proxies\n- Connection management requires heartbeats, reconnection logic, and careful error handling\n\n### Which Should You Choose?\n\n\n| Approach | Latency | Overhead | Bidirectional | Best For |\n|----------|---------|----------|---------------|----------|\n| **Long Polling** | High | High | No | Legacy systems, fallback |\n| **SSE** | Medium | Low | No | Notifications, live feeds |\n| **WebSocket** | Lowest | Lowest | Yes | Chat, gaming, collaboration |\n\n\n**For a messaging system, WebSocket is the clear winner.** The bidirectional nature matches how chat works: both users send and receive constantly. The low latency means conversations feel instant. The single-connection efficiency means we can handle more users per server.\n\nThe main challenge with WebSocket is the stateful nature, but we have already addressed this with our Session Service design. We accept the added complexity because the user experience benefits are substantial.\n\n\n> **One practical note**\n>\n> Always implement long polling as a fallback. Some corporate networks and older proxies still block WebSocket connections. Your client should detect this and gracefully fall back to long polling\n\n\n---\n\n## 6.2 Message Delivery Guarantees\n\nEveryone who has used WhatsApp knows the checkmarks: one gray for sent, two gray for delivered, two blue for read. These simple icons hide a lot of complexity. \n\nHow do we track message state reliably across unreliable networks, flaky mobile connections, and devices that go offline unpredictably?\n\nLet's break down what each state means and how we guarantee correct transitions:\n\n1. **Sent (single gray checkmark):** The message has been persisted on our servers. The user can close the app knowing the message will not be lost.\n2. **Delivered (double gray checkmarks):** The message has reached the recipient's device. They may not have seen it yet, but it is on their phone.\n3. **Read (double blue checkmarks):** The recipient has opened the conversation and viewed the message.\n\nGetting these states right requires careful engineering. Networks fail, devices go offline mid-delivery, and the same message might be sent twice due to retries. Let's see how to handle this.\n\n### The Delivery Flow\n\n\n```mermaid\nsequenceDiagram\n participant A as User A\n participant S as Server\n participant B as User B\n\n A->>S: Send \"Hello!\"\n Note over S: Store in database\n S-->>A: ACK (message_id: 123)\n Note over A: Show ✓ (Sent)\n\n S->>B: Push \"Hello!\"\n B-->>S: ACK received\n Note over S: Update status\n S-->>A: Status: delivered\n Note over A: Show ✓✓ (Delivered)\n\n Note over B: User opens conversation\n B->>S: Mark 123 as read\n Note over S: Update status\n S-->>A: Status: read\n Note over A: Show ✓✓ (Read/Blue)\n```\n\n\n### Ensuring At-Least-Once Delivery\n\nThe cardinal rule of messaging: **messages must never be lost.** A user who sees the \"sent\" checkmark should be confident that their message will eventually reach its destination, even if networks fail, servers crash, or the recipient's phone runs out of battery.\n\nAchieving this requires a combination of two techniques: the client retries aggressively, and the server deduplicates.\n\n#### **Client-Side Retry with Idempotency**\n\nHere is the key insight that makes reliable messaging possible: if the server can detect duplicate messages, the client can safely retry as many times as needed without fear of the message appearing twice.\n\n\n```mermaid\nsequenceDiagram\n participant C as Client\n participant S as Server\n participant DB as Database\n\n Note over C: Generate client_message_id: \"abc-123\"\n\n C->>S: Send (client_id: abc-123)\n Note over C,S: Network timeout - no response\n\n C->>S: Retry (client_id: abc-123)\n S->>DB: Does abc-123 exist?\n DB-->>S: No\n S->>DB: Store message\n S-->>C: ACK (stored as msg_456)\n\n C->>S: Retry again (client_id: abc-123)\n S->>DB: Does abc-123 exist?\n DB-->>S: Yes, already stored as msg_456\n S-->>C: ACK (already stored as msg_456)\n Note over S: Deduplicated!\n```\n\n\nHere is how this works in practice:\n\n1. Before sending, the client generates a unique `client_message_id` (typically a UUID). This ID is the message's fingerprint.\n2. The client sends the message to the server and starts a timer.\n3. If no acknowledgment arrives within a timeout (say, 5 seconds), the client assumes something went wrong and retries with the exact same `client_message_id`.\n4. When the server receives a message, it checks: \"Have I seen this `client_message_id` before?\"\n5. If yes, it is a duplicate. The server returns success without storing again.\n6. If no, it is a new message. The server stores it and returns success.\n\nThis pattern is called **idempotent delivery**. The same operation can be performed multiple times with the same result. The client can retry as aggressively as it needs to, and the server guarantees that duplicate messages are detected and discarded.\n\n#### **Server-Side Persistence Before Acknowledgment**\n\nThere is one more critical rule for reliable messaging: **never acknowledge a message until it is persisted to durable storage.**\n\n\n```mermaid\nflowchart LR\n A[Receive Message]:::primary --> B[Write to Database]:::secondary\n B --> C{Write Success?}:::orange\n C -->|Yes| D[Send ACK to Client]:::green\n C -->|No| E[Return Error]:::red\n E --> F[Client Retries]:::primary\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef secondary fill:#38d9a9,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n```\n\n\nIf the server crashes between receiving a message and persisting it, the message is lost. By only sending ACK after persistence, we guarantee that any acknowledged message is safely stored.\n\n### Handling Out-of-Order Messages\n\nNetworks don't guarantee ordering. If User A sends \"Hello\" then \"How are you?\", network conditions might deliver them in reverse order.\n\n\n```mermaid\nflowchart LR\n subgraph Sent[\"Sent Order\"]\n S1[\"Msg 1: Hello\"]:::primary\n S2[\"Msg 2: How are you?\"]:::primary\n end\n\n subgraph Network[\"Network Chaos\"]\n N[Packets take
different routes]:::orange\n end\n\n subgraph Received[\"Received Order\"]\n R1[\"Msg 2: How are you?\"]:::red\n R2[\"Msg 1: Hello\"]:::red\n end\n\n subgraph Display[\"Correct Display\"]\n D1[\"Msg 1: Hello\"]:::green\n D2[\"Msg 2: How are you?\"]:::green\n end\n\n S1 & S2 --> N --> R1 & R2\n R1 & R2 --> SORT[Sort by
sequence number]:::secondary\n SORT --> D1 & D2\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef secondary fill:#38d9a9,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n```\n\n\nThe solution involves multiple mechanisms:\n\n1. **Sequence numbers per conversation:** Each message gets an incrementing sequence number within its conversation\n2. **Server-side timestamp:** Server assigns authoritative timestamp for ordering\n3. **Client-side reordering:** Client sorts messages by sequence number before display\n\nWhen the client receives a message, it doesn't immediately display it. Instead, it inserts it into the correct position based on sequence number, ensuring messages always appear in order regardless of arrival order.\n\n---\n\n## 6.3 Presence System (Online/Offline Status)\n\nThe green \"online\" dot and \"last seen at 3:45 PM\" text seem like simple features. But think about what they require at scale: tracking 50 million concurrent users, notifying their contacts when status changes, and doing it all without overwhelming the system.\n\nThis is a classic trade-off between accuracy and efficiency. Perfect real-time presence would require broadcasting every status change to potentially hundreds of contacts, generating massive network traffic. We need a smarter approach.\n\n### The Challenges\n\nThe core challenges with presence are:\n\n- **Constant churn:** Users open and close apps constantly. Their phones switch between WiFi, cellular, and airplane mode. Status changes happen all the time.\n- **Fanout explosion:** If User A has 500 contacts and comes online, do we notify all 500? What if 1000 users come online in the same second?\n- **Accuracy vs. efficiency:** Users want accurate presence, but broadcasting every change would overwhelm the system.\n\n### Approach: Heartbeat-Based Presence with Lazy Queries\n\nThe practical solution is a combination of heartbeats for tracking and lazy queries for display.\n\n\n```mermaid\nsequenceDiagram\n participant C as Client\n participant CS as Chat Server\n participant R as Redis\n\n C->>CS: Connect + \"I'm online\"\n CS->>R: SET presence:user123 \"online\"
EXPIRE 30 seconds\n\n loop Every 10 seconds\n C->>CS: Heartbeat (I'm still here)\n CS->>R: SET presence:user123 \"online\"
EXPIRE 30 seconds\n end\n\n Note over C: User closes app\n Note over C: No more heartbeats\n\n Note over R: 30 seconds pass...\n Note over R: Key expires automatically\n Note over R: User is now \"offline\"\n```\n\n\n#### How It Works\n\nThe mechanism is elegantly simple:\n\n1. **Connection:** When a user connects, the chat server sets a key in Redis: `presence:user_123 = online` with a TTL of 30 seconds.\n2. **Heartbeat:** Every 10 seconds, the client sends a heartbeat over the WebSocket. The server refreshes the TTL, resetting it to 30 seconds.\n3. **Disconnect:** If heartbeats stop (the user closed the app, lost network, or the phone died), the key expires automatically after 30 seconds.\n4. **Query:** When User B opens a chat with User A, the app queries: \"What is the presence of User A?\" Redis returns the value if the key exists, or nothing if it has expired.\n\nThe 30-second TTL is a deliberate choice. It means users appear offline within 30 seconds of actually going offline, which is acceptable for casual chat. If you needed faster detection (for a stock trading app, say), you could reduce the TTL and heartbeat interval, at the cost of more traffic.\n\n#### Optimizing Presence Fanout\n\nThe naive approach, broadcasting presence changes to all contacts, doesn't scale. If a user has 500 contacts, and 10% of users change presence every minute, the fanout traffic explodes.\n\n**Solution: Lazy Presence Queries**\n\nInstead of broadcasting, query presence only when needed:\n\n\n```mermaid\nflowchart LR\n subgraph Trigger[\"User Opens Chat\"]\n A[\"User A opens chat
with User B\"]:::primary\n end\n\n subgraph Query[\"Lazy Query\"]\n Q[\"GET presence:user_b\"]:::secondary\n end\n\n subgraph Redis[\"Redis\"]\n R[(\"user_b: online
TTL: 25s remaining\")]:::purple\n end\n\n subgraph UI[\"Update UI\"]\n D[\"Show: User B is online 🟢\"]:::green\n end\n\n A --> Q --> R --> D\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef secondary fill:#38d9a9,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\nWhen User A opens a chat with User B:\n\n1. Query Redis for User B's presence\n2. Display result in UI\n3. Subscribe to presence updates (via Redis pub/sub) for real-time changes while chat is open\n4. Unsubscribe when chat is closed\n\nThis drastically reduces presence traffic. We only track presence for users the client is actively viewing.\n\n### Last Seen Timestamp\n\nInstead of binary online/offline, many apps show \"last seen at [time]\":\n\n1. Update `last_seen` timestamp on every meaningful user action\n2. When queried, return the timestamp\n3. Client displays relative time (\"last seen 5 minutes ago\")\n\nThis provides useful information without the complexity of real-time presence. WhatsApp uses this approach, only showing \"online\" status for users you're actively chatting with.\n\n---\n\n## 6.4 Message Synchronization Across Devices\n\nModern users expect their messages on every device: phone, tablet, laptop, web browser. When they read a message on their phone, it should show as read on their laptop too. This is multi-device sync.\n\n### The Challenge\n\n\n```mermaid\nflowchart LR\n subgraph UserA[\"User A's Devices\"]\n Phone[\"Phone
Online\"]:::green\n Tablet[\"Tablet
Offline\"]:::red\n Web[\"Web Browser
Online\"]:::green\n end\n\n subgraph Server[\"Message Arrives\"]\n MSG[\"New message
from User B\"]:::primary\n end\n\n MSG --> Phone\n MSG --> Web\n MSG -.->|\"Queue for later\"| Tablet\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n```\n\n\nWhen a message arrives for User A, we need to:\n\n1. Push to all online devices immediately\n2. Queue for offline devices\n3. Sync read/delivered status across all devices\n\n### Hybrid Sync Strategy\n\nThe best approach combines real-time push with catch-up pull:\n\n#### Real-time Push (Online Devices)\n\nWhen User A has multiple devices connected, the Session Service tracks all of them:\n\n\n```shell\nuser_a -> [chat_server_1 (phone), chat_server_3 (web)]\n```\n\n\nWhen a message arrives:\n\n1. Session Service returns all device connections\n2. Message is pushed to all connected devices simultaneously\n3. Each device sends independent ACK\n4. \"Delivered\" status is set when ANY device acknowledges\n\n#### Catch-up Pull (Reconnecting Devices)\n\nWhen a device comes online after being offline:\n\n\n```mermaid\nsequenceDiagram\n participant T as Tablet (was offline)\n participant CS as Chat Server\n participant MS as Message Service\n participant MQ as Message Queue\n\n T->>CS: Connect (last_sync: 2 hours ago)\n CS->>MS: Get messages since last_sync\n MS-->>CS: 47 new messages\n CS->>MQ: Any queued messages?\n MQ-->>CS: 3 queued messages\n CS->>T: Here are 50 messages\n T-->>CS: ACK all received\n CS->>MQ: Clear queue for this device\n```\n\n\nThe device sends its last sync timestamp when connecting. The server fetches all messages since then and delivers them in bulk. This ensures no messages are ever missed, regardless of how long the device was offline.\n\n### Read Status Synchronization\n\nWhen User A reads a message on their phone:\n\n\n```mermaid\nsequenceDiagram\n participant P as Phone\n participant S as Server\n participant W as Web Browser\n participant T as Tablet\n\n Note over P: User reads message\n P->>S: Mark message 456 as read\n S->>S: Update database\n\n par Push to sender\n S->>S: Notify User B: message read\n and Sync to other devices\n S->>W: Sync: message 456 read\n S->>T: Sync: message 456 read\n end\n\n Note over W: UI updates: message shown as read\n Note over T: UI updates: message shown as read\n```\n\n\nAll of User A's devices see the same read status. The sender (User B) also gets notified that the message was read.\n\n---\n\n## 6.5 Scaling Chat Servers\n\nChat servers are fundamentally different from typical web servers. While a stateless API server can be scaled by simply adding more instances behind a load balancer, chat servers hold state: the WebSocket connections themselves. \n\nEach connection represents a user, and that user's messages must be routed to their specific server. This stateful nature creates unique scaling challenges. \n\nLet's walk through how to handle them.\n\n### Connection Limits\n\nA well-tuned server with proper kernel configuration can handle 50,000 to 100,000 concurrent WebSocket connections. The limits come from file descriptors, memory, and CPU for processing messages.\n\nFor our target of 50 million concurrent users, we need:\n\n\n```shell\n50M connections / 50K per server = 1,000 chat servers\n```\n\n\n\n```mermaid\nflowchart TB\n subgraph Users[\"50 Million Users\"]\n U[\"All Users\"]:::primary\n end\n\n subgraph LB[\"Load Balancing Layer\"]\n L[\"Consistent Hashing
by user_id\"]:::orange\n end\n\n subgraph Servers[\"1,000 Chat Servers\"]\n S1[\"Server 1
50K connections\"]:::secondary\n S2[\"Server 2
50K connections\"]:::secondary\n S3[\"Server 3
50K connections\"]:::secondary\n S1000[\"Server 1000
50K connections\"]:::secondary\n end\n\n U --> L\n L --> S1 & S2 & S3 & S1000\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef secondary fill:#38d9a9,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n```\n\n\n### Sticky Sessions and Connection Affinity\n\nUnlike stateless HTTP servers, WebSocket connections are inherently stateful. Once User A connects to Chat Server 1, all their messages must route through that server until they disconnect.\n\n**Load balancer configuration options:**\n\n1. **Consistent hashing by user_id:** Same user always routes to the same server (until server list changes)\n2. **Connection tracking:** Load balancer remembers which server each connection went to\n3. **Client-side server assignment:** Server tells client which specific server to connect to on subsequent attempts\n\n### Handling Server Failures\n\nWhat happens when a chat server crashes? With 50,000 users per server, a crash is a significant event.\n\n\n```mermaid\nsequenceDiagram\n participant C as Client\n participant S1 as Server 1\n participant LB as Load Balancer\n participant S2 as Server 2\n participant SS as Session Service\n participant MQ as Message Queue\n\n C->>S1: Connected and chatting...\n Note over S1: Server crashes!\n\n C--xS1: Connection lost\n Note over C: Detect disconnect
(no heartbeat response)\n\n C->>LB: Reconnect attempt\n LB->>S2: Route to healthy server\n C->>S2: Establish new connection\n\n S2->>SS: Update: User now on Server 2\n S2->>MQ: Fetch pending messages\n MQ-->>S2: 3 messages queued during crash\n S2->>C: Deliver missed messages\n\n Note over C: Back online, no messages lost\n```\n\n\nThe recovery flow:\n\n1. **Detection:** Client detects disconnection (heartbeat timeout, connection close event)\n2. **Reconnection:** Client connects to load balancer, which routes to a healthy server\n3. **State Recovery:** New server registers the connection in Session Service\n4. **Message Recovery:** Pending messages are fetched from the queue\n5. **Resume:** Normal operation continues\n\nThe key insight is that message persistence (in the database and queue) is separate from connection state. Even if a server crashes mid-delivery, the message is safe and will be delivered on reconnection.\n\n### Graceful Shutdown\n\nProduction systems need regular maintenance: OS patches, code deployments, hardware replacements. Graceful shutdown minimizes user impact:\n\n1. **Mark as draining:** Remove server from load balancer pool\n2. **Stop new connections:** Reject any new WebSocket handshakes\n3. **Notify clients:** Send a \"please reconnect elsewhere\" message\n4. **Wait for drain:** Give clients time to gracefully disconnect (typically 30-60 seconds)\n5. **Force close:** Terminate any remaining connections\n6. **Shutdown:** Server can now safely restart\n\nMost clients will reconnect to other servers during the drain period, making the maintenance nearly invisible to users.\n\n---\n\n## 6.6 End-to-End Encryption (Conceptual)\n\nEnd-to-end encryption (E2EE) ensures that only the sender and recipient can read messages. Even the service provider (WhatsApp, Signal, etc.) cannot decrypt message content.\n\n### How It Works (Signal Protocol)\n\nMost modern messaging apps use the Signal Protocol or something similar:\n\n\n```mermaid\nsequenceDiagram\n participant A as User A\n participant S as Server\n participant B as User B\n\n Note over A: Generate key pair
(public_A, private_A)\n Note over B: Generate key pair
(public_B, private_B)\n\n A->>S: Register my public_A\n B->>S: Register my public_B\n\n Note over A: Want to message User B\n A->>S: Get public_B\n S-->>A: Here's public_B\n\n Note over A: Encrypt \"Hello!\" with public_B\n A->>S: Send encrypted message\n\n Note over S: I can store this, but
I cannot read it!\n\n S->>B: Forward encrypted message\n Note over B: Decrypt with my private_B\n Note over B: \"Hello!\"\n```\n\n\nThe basic flow:\n\n1. **Key Generation:** Each user's device generates a public/private key pair\n2. **Key Registration:** Public keys are uploaded to the server\n3. **Key Exchange:** When starting a conversation, users fetch each other's public keys\n4. **Encryption:** Sender encrypts message using recipient's public key\n5. **Transmission:** Encrypted message travels through servers (who can't decrypt it)\n6. **Decryption:** Only recipient's private key can decrypt the message\n\n### What the Server Can and Cannot Do\n\n\n```mermaid\nflowchart LR\n\n subgraph Cannot[\"Server CANNOT\"]\n X1[\"Read message
content\"]:::red\n X2[\"Decrypt any
messages\"]:::red\n X3[\"Provide content
to third parties\"]:::red\n X4[\"Search message
content\"]:::red\n end\n\n subgraph Can[\"Server CAN\"]\n C1[\"Route encrypted
messages\"]:::green\n C2[\"Store encrypted
data\"]:::green\n C3[\"Track delivery
status\"]:::green\n C4[\"Send push
notifications\"]:::green\n end\n\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n```\n\n\n### Trade-offs of E2E Encryption\n\n**Benefits:**\n\n- Strong privacy protection\n- Users trust the system more\n- Regulatory compliance in some regions\n\n**Challenges:**\n\n- **Multi-device complexity:** Each device has its own key pair. Syncing messages across devices requires encrypting for each device separately.\n- **Key changes:** If a user reinstalls the app or gets a new phone, their keys change. The system must handle this securely without enabling man-in-the-middle attacks.\n- **Limited server features:** Search, spam detection, and content moderation become difficult or impossible when the server can't read content.\n- **Backup challenges:** If users back up their messages, the backup is also encrypted. Losing the key means losing access to message history.\n\nFor an interview, it's sufficient to mention that E2EE is important for privacy and explain the high-level concept. The cryptographic details (perfect forward secrecy, double ratchet algorithm, etc.) are typically out of scope unless the interviewer specifically asks.\n\n---\n\n# Quiz","pageType":"system-design-interviews"}

Design WhatsApp

Ashish Pratap Singh

Get Premium