Every ML problem starts with a simple question: do you have labeled data or not?

That choice shapes everything that follows. It determines which algorithms you can use, how you evaluate the model, and how you design the data pipeline.

This chapter introduces the four main learning paradigms and builds intuition for when to use each.

Supervised learning is the most commonly used approach in real-world ML systems.

You start with examples where the correct answer is already known. The model learns by looking at these input–output pairs and figuring out how to map one to the other. Once trained, it can take new inputs and make predictions it hasn’t seen before.

The term “supervised” comes from the idea of a teacher guiding the process. Each training example includes:

The model’s job is to learn patterns in the inputs that consistently lead to the right output.

Supervised learning typically falls into two categories, depending on what you’re trying to predict:

The next chapter dives deep into classification and regression, including multi-class and multi-label variants. For now, the distinction is straightforward: discrete categories vs. continuous numbers.

Why Supervised Learning Dominates Production

The reason supervised learning powers the majority of production ML systems is simple: when you have labels, you can measure exactly how wrong your model is. That measurability makes everything easier. You can compare models objectively, track improvements over iterations, and set clear deployment thresholds.

Consider how these real systems rely on supervised learning:

Across all of these, the pattern is the same: clear inputs, clear outputs, and a way to measure performance.

The Label Problem

The same thing that makes supervised learning powerful also makes it hard: you need labels, and labels are often expensive. Someone (or something) has to provide the correct answer for every training example.

Some domains get labels for free. E-commerce recommendation systems know whether a user clicked or purchased. Ad systems know whether someone converted. These are called implicit labels, they come from user behavior rather than human annotation.

Other domains require explicit labeling. Medical image classification needs radiologists to annotate scans. Content moderation needs human reviewers to flag policy violations. Self-driving car perception models need humans to draw bounding boxes around every object in every frame.

Unsupervised learning works with unlabeled data. There's no "correct answer" to learn from. Instead, the model looks for structure, patterns, and groupings that exist naturally in the data.

This sounds less useful than supervised learning, but it solves a different class of problems entirely. When you don't know what you're looking for, or when labeling millions of data points isn't feasible, unsupervised learning helps you discover patterns you might not have considered.

Unsupervised learning is usually applied in three main ways, each solving a different kind of problem:

Clustering groups similar data points together. Spotify uses clustering to create listener segments with similar music preferences. E-commerce platforms cluster products by purchasing patterns rather than manually assigned categories. The algorithm doesn't know what the groups "mean", it just finds them based on similarity in the feature space.

Dimensionality reduction compresses high-dimensional data into fewer dimensions while preserving meaningful structure. If you have user profiles with 500 features, dimensionality reduction might compress them to 50 features that capture the same information. This is useful both as a preprocessing step (reducing computational cost for downstream models) and for visualization (plotting high-dimensional data in 2D to spot patterns).

Anomaly detection identifies data points that don't fit the normal pattern. Credit card companies use unsupervised anomaly detection to flag unusual transactions. Network security systems detect intrusion attempts by identifying traffic patterns that deviate from the baseline. The advantage over supervised fraud detection: you don't need examples of every type of fraud, you just need to know what "normal" looks like.

Trade-offs vs. Supervised Learning

Unsupervised learning avoids the labeling bottleneck, but it comes with its own challenges. The biggest one: evaluation is harder. With supervised learning, you compare predictions to known labels and compute accuracy, precision, or AUC. With unsupervised learning, there's often no objective "right answer" to compare against.

Did your clustering algorithm find 5 customer segments or 8? Which is correct? That depends on how useful the segments are for downstream business decisions, not on a mathematical ground truth. This subjectivity makes unsupervised models harder to evaluate and iterate on.

In practice, many production systems use unsupervised learning as a component within a larger supervised pipeline. Clustering creates features that feed into a supervised model. Dimensionality reduction preprocesses data before classification. Anomaly detection flags candidates that a supervised model then confirms or rejects.

Labels are expensive. Unlabeled data is cheap. Semi-supervised learning exploits this gap by using a small amount of labeled data alongside a large amount of unlabeled data.

The core idea: if you have 1,000 labeled examples and 1,000,000 unlabeled examples, a semi-supervised approach can often outperform a supervised model trained on just the 1,000 labeled examples. The unlabeled data helps the model understand the underlying structure of the input space, even though it doesn't have labels for most of it.

A concrete example: content moderation at scale. A platform like YouTube receives hundreds of millions of new videos and comments daily. Human reviewers can label maybe tens of thousands of items per day, covering a tiny fraction of the total. A semi-supervised approach works like this:

Label propagation is one common technique. If a labeled example is "toxic" and ten unlabeled examples are very similar to it in the feature space, those ten are likely toxic too. The algorithm propagates labels through the similarity graph, expanding the effective training set.

Semi-supervised learning works best when:

It works poorly when the unlabeled data is noisy, unrelated to the task, or when the labeled set is so small that it doesn't capture the real class distribution.

Self-supervised learning takes a different approach to the label problem: instead of getting labels from humans or from user behavior, it creates labels from the data itself.

The trick is to design a pretext task where the input provides its own supervision. Take a sentence and mask a word. The model's job is to predict the missing word. No human annotation needed. The "label" is just the original word that was masked.

This is exactly how BERT works. It masks 15% of words in a sentence and trains to predict them. In doing so, it learns deep representations of language, word meanings, grammar, and context, all without a single human label. Those representations then transfer to downstream tasks like search ranking, sentiment analysis, or question answering through fine-tuning.

Self-supervised learning has become the foundation of modern ML, particularly for unstructured data:

The key insight for system design: self-supervised pretraining on large unlabeled datasets, followed by fine-tuning on small labeled datasets, often beats supervised training on the labeled data alone. This is why foundation models (BERT, GPT, CLIP) have changed how teams build ML systems. Instead of training from scratch, you start with a pretrained model and adapt it to your specific task with far less labeled data.

The decision comes down to three factors: what labels you have, how much data you have, and what problem you're solving.

Here's how the four approaches compare across key dimensions:

A few rules of thumb for interviews:

In practice, production systems rarely use just one approach. A recommendation system might use self-supervised pretraining to learn item embeddings, unsupervised clustering to group similar items, and supervised learning to rank candidates for a specific user. The paradigms complement each other.