Design a Video Recommendation System
A video recommendation system decides which videos to show each user without the user searching for anything. It learns from viewing behavior and video content to retrieve a small set of promising candidates from a huge catalog, ranks them by predicted value to the user, and assembles a diverse, eligible list for surfaces like the homepage or the Up Next panel.
Imagine opening YouTube after a long day.
You haven't searched for anything. You haven't told the app what you want to watch. Yet the homepage is already filled with videos that seem surprisingly relevant to you. Maybe there's a new video from a creator you follow. A tutorial related to something you were learning yesterday. Or a completely new topic that you didn't even know you'd find interesting.
Now imagine doing this for hundreds of millions of users, each with different interests, preferences, and viewing habits. And imagine choosing those recommendations from a catalog containing hundreds of millions of videos.
That's the challenge we're going to solve.
In this article, we'll design a large-scale video recommendation system, similar to the one used by platforms like YouTube. But we'll approach it as a machine learning system design interview.
We'll start by clarifying the requirements and defining what success actually means. Then we'll explore how to collect training data, represent users and videos, retrieve relevant candidates, and rank them using machine learning. We'll also cover cold starts, real-time personalization, experimentation, and the infrastructure needed to make everything work at scale.
By the end, you'll understand not just the architecture of a video recommendation system, but also the decisions, trade-offs, and reasoning behind each part of the design.