Design Live Comments
What are Live Comments?
Live comments are a feature often used in streaming platforms, sports apps, and social media events where users can post and view comments in real time as an event unfolds.
For example, during a live football match or a concert stream, thousands or even millions of viewers may send comments simultaneously, which appear instantly for all participants.
In this chapter, we will aim to design a low-latency, scalable live comment system that allows thousands or even millions of users to exchange messages in real time during a live event.
Key challenges include:
- Latency: Messages must be delivered in near real-time.
- Fanout: A single message needs to be broadcast to millions of subscribers simultaneously.
- Ordering: Comments should appear in a logical, roughly chronological order.
Let’s begin by clarifying the requirements.
1. Clarifying Requirements
Before diving into the design, let's narrow down the scope of the problem. Here’s an example of how a discussion between candidate and interviewer might flow:
Candidate: “Should users only post text comments, or do we also need to support images, emojis, or reactions?”
Interviewer: “For now, only short text-based comments. Reactions or media attachments can be ignored.”
Candidate: “Do comments need to be delivered in real-time to all viewers?”
Interviewer: “Yes, the experience should feel live. Ideally under 500 ms from publish to delivery.”
Candidate: “Do we need to support playback after the live event ends like showing comments alongside recorded video?”
Interviewer: “Yes. Playback is in scope. Users watching later should see the comments synchronized with the original timeline.”
Candidate: “Can users reply to comments or have threaded conversations?”
Interviewer: “No. No replies or threads. Just a flat stream of comments.”
Candidate: “What about comment moderation like spam filtering, profanity, abuse detection?”
Interviewer: “Assume moderation already exists. You can treat it as out of scope.”
Candidate: “For ordering, do we need strict global time ordering across distributed regions?”
Interviewer: “No. Exact strict ordering is not require. We just need roughly correct chronological order that feels consistent to the user.”
After clarifying the requirements, we can summarize the functional and non-functional requirements.
1.1 Functional Requirements
- Post Comments: Users can post short, text-based comments during a live event.
- Real-Time Viewing: Users can view new comments posted by others in real-time.
- Playback: Users can replay the stream with synchronized comments (time-aligned with the video) after the event ends.
Out of Scope:
- Reactions: Users can react (like, heart, etc.) to comments.
- Replies: Users can reply to comments.
- Moderation: The system must filter spam, and other undesirable content.
1.2 Non-Functional Requirements
- High Scalability: Support millions of concurrent viewers and thousands of new comments per second.
- Low Latency: Deliver comments to clients within sub-second latency (< 500 ms ideally).
- Reliability & Durability: Once accepted, a comment should never be lost even during failures.
- Ordering: Comments should appear in approximately the order they were sent. Strict ordering is difficult in a distributed system, but we should aim for a consistent experience.
- Eventual Consistency: Small ordering differences or delays are acceptable as long as the user experience remains smooth.
2. Scale Estimation
Before designing the system, let’s estimate the scale we need to support.
Assumptions:
- Concurrent viewers: 5 million. This reflects a peak load during major live events (e.g., sports finals, global product launches).
- Active commenters: 1% of viewers (≈50,000 users). Most users are passive viewers; only a small fraction actively comment.
- Comment rate per user: 1 comment every 10 seconds during periods of high activity.
- Average comment size: 150 bytes. This includes the comment text plus metadata (user ID, timestamp, event ID).
Throughput Estimation
- Incoming comments per second = 50,000 users × (1/10) = 5,000 comments/sec
- Outgoing fanout = 5,000 comments/sec × 5,000,000 recipients = 25 billion message deliveries/sec. This massive number is the core challenge and will be handled via a distributed fanout architecture, not a single process.
Storage Estimation
- Storage rate: 5,000 comments/sec × 150 bytes/comment = 0.75 MB/sec.
- Storage per day: 0.75 MB/sec × 86,400 seconds/day ≈ 65 GB/day.
- Storage per month (for playback): 65 GB/day × 30 days ≈ ~2 TB/month.