The previous chapter covered the four learning paradigms. Within supervised learning, the most common paradigm in production, every prediction task falls into one of two categories: classification or regression. This choice affects your model architecture, loss function, evaluation metrics, and serving infrastructure.

Choosing the wrong framing does not just lead to a suboptimal model, it means optimizing for the wrong objective from the start.

Classification predicts a discrete category. Given an input, the model assigns it to one of a fixed set of classes.

Binary classification is the simplest and most common variant: two possible outcomes. Is this email spam or legitimate? Is this transaction fraudulent or genuine? Will this user click on this ad or not? These yes/no decisions power some of the highest-impact ML systems in production.

But classifiers don't actually output a hard yes or no. They output a probability, a score between 0 and 1 representing the model's confidence that the input belongs to the positive class. A fraud detection model might output 0.87 for a suspicious transaction, meaning "87% chance this is fraud." The system then applies a threshold to convert that probability into a decision.

The threshold is a design decision, not a fixed rule. Setting it at 0.5 seems natural, but it's rarely optimal. Consider fraud detection: a false negative (missing real fraud) costs the company money and erodes user trust.

A false positive (blocking a legitimate transaction) frustrates the user but causes less harm. To catch more fraud, you lower the threshold to 0.3, which flags more transactions as fraudulent. You catch more actual fraud, but you also block more legitimate transactions.

This is the precision-recall tradeoff. Lower thresholds increase recall (catching more positives) at the cost of precision (more false alarms). Higher thresholds increase precision at the cost of recall. The right threshold depends on the business context, not on mathematics.

This threshold tuning is one reason why classification systems need calibrated probabilities. If the model says 0.7, that should mean roughly 70% of such predictions are actually positive. Poorly calibrated models make threshold selection unreliable.

Binary classification handles two outcomes. Multi-class classification extends this to N mutually exclusive categories, where each input belongs to exactly one class.

Language detection is a clean example. Given a text snippet, the model predicts one of 100+ languages. Product categorization works the same way: an e-commerce listing gets assigned to one category from a taxonomy of thousands. Image classification labels a photo as one of many object types.

The key word is mutually exclusive. Each input gets exactly one label. A product can't be both "Electronics" and "Kitchen Appliances" in a strict multi-class setup. If it can, that's a multi-label problem (covered in the next section).

Mechanically, multi-class models output a probability distribution across all classes using a softmax function. The probabilities sum to 1, and the class with the highest probability becomes the prediction.

Multi-class classification gets interesting at scale. A few real-world examples:

When the number of classes grows into the thousands, standard softmax becomes expensive because it computes a score for every class on every prediction. Production systems at that scale use techniques like hierarchical softmax or sampled softmax to reduce computation. A later chapter on algorithms covers these techniques in more detail.

One subtlety worth noting: multi-class problems sometimes have classes that aren't equally important. In a content safety system with categories like "safe", "mildly offensive", "hate speech", and "illegal content", misclassifying illegal content as safe is far worse than the reverse. The evaluation metrics and loss function weights need to reflect these asymmetric costs.

Multi-label classification allows each input to have multiple labels simultaneously. A movie can be both "Action" and "Comedy." A news article can cover "Politics," "Economics," and "Healthcare." A social media post can violate multiple content policies at once.

The architectural difference from multi-class is fundamental. Multi-class uses softmax, which forces probabilities to sum to 1 (the classes compete with each other). Multi-label uses independent sigmoid functions, one per label. Each label gets its own probability between 0 and 1, and each label has its own threshold.

This distinction has real consequences for system design:

The "label correlation" row matters more than it might seem. In multi-label problems, labels are often correlated. Movies tagged as "Horror" are more likely to also be tagged as "Thriller" than "Comedy."

Content that violates "hate speech" policies often also violates "harassment" policies. Naive multi-label models with independent sigmoids ignore these correlations. More sophisticated approaches model label dependencies using techniques like label-aware attention or graph neural networks over the label taxonomy.

Systems like YouTube's video tagging, Google's image labeling, and Stack Overflow's question tagging all use multi-label classification. At YouTube's scale, a single video might receive 5-15 tags from a vocabulary of thousands, and the system needs to handle tag correlations, tag hierarchy, and evolving tag vocabularies.

Regression predicts a continuous numeric value. Instead of assigning inputs to categories, the model outputs a number on a continuous scale.

Regression powers many of the ML systems people interact with daily. Uber's ETA prediction estimates arrival time in minutes. Zillow's Zestimate predicts home values in dollars. Google Ads' bid optimizer predicts the expected revenue from showing a specific ad. YouTube's watch-time prediction estimates how many seconds a user will watch a video (a key signal for ranking).

Unlike classification, regression outputs don't have a natural decision boundary. There's no threshold to tune. But regression has its own design considerations:

Bounded vs. unbounded outputs

Some predictions have natural bounds: a probability score must be between 0 and 1, a rating between 1 and 5, delivery time can't be negative. Others are theoretically unbounded: revenue predictions, price estimates, or time-to-event.

Bounded outputs often benefit from output transformations (sigmoid for 0-1, clipping for hard bounds). Unbounded outputs need careful handling of outliers during training, since a few extreme values can dominate the loss function.

Point predictions vs. uncertainty estimates

A model that predicts "delivery in 23 minutes" is useful. A model that predicts "delivery in 23 minutes, 95% likely between 18 and 31 minutes" is more useful. Many production regression systems output prediction intervals or full probability distributions rather than single point estimates.

Uber shows an ETA range for exactly this reason, a single number creates false precision.

Regression evaluation uses different metrics than classification. Instead of accuracy or precision, regression models are evaluated by how far off their predictions are: mean absolute error (MAE), mean squared error (MSE), or root mean squared error (RMSE).

The choice between these depends on how much you want to penalize large errors. MSE penalizes large errors quadratically, making it sensitive to outliers. MAE treats all errors linearly. A later chapter covers loss functions and evaluation metrics in detail.

The two task types differ across several dimensions:

When the Boundary Blurs

The line between classification and regression isn't always clean. Several common scenarios sit in the gray area:

Ordinal classification

Star ratings (1-5 stars), severity levels (low/medium/high/critical), and customer satisfaction scores (1-10) have a natural ordering. You could treat these as multi-class classification (ignoring the order) or as regression (treating them as continuous). Neither is perfect.

Multi-class classification ignores that predicting 4 when the answer is 5 is better than predicting 1. Regression assumes equal intervals between values, which is rarely true (the difference between 1-star and 2-stars isn't the same as between 4-stars and 5-stars).

Discretized regression

Sometimes a regression problem is easier to solve as classification. Predicting exact house prices is hard. Predicting price buckets ($0-200K, $200K-400K, $400K-600K, etc.) is easier and might be sufficient for the use case. Uber's surge pricing model doesn't predict an exact multiplier. It predicts a bucket (1x, 1.2x, 1.5x, 2x), which is effectively classification over a set of multiplier tiers.

Classification with probability outputs used as scores

Click-through rate prediction is technically binary classification (clicked or didn't), but the output probability itself is the valuable signal. Ad ranking systems use the predicted click probability directly as a score to rank ads, not as a yes/no decision. The model is a classifier by architecture, but the system uses it as a regression model in practice.

The same real-world problem can often be framed either way. The right framing depends on what the system needs to do with the prediction.

A few principles guide the decision:

Start with the downstream action

If the system needs to make a binary or categorical decision (block this content, approve this loan, route this ticket), classification is natural. If the system needs a magnitude (how long, how much, how many), regression fits better.

Consider what information matters

Binary classification collapses information. "This user will engage" throws away whether they'll engage for 5 seconds or 50 minutes. If that difference matters for ranking or business decisions, regression preserves it.

Think about evaluation

Classification is easier to evaluate with stakeholders. "We catch 95% of fraud" is intuitive. "Our RMSE is 3.2 minutes" requires more context to interpret. If you need to communicate model quality to non-technical stakeholders, classification metrics are more accessible.

Consider label availability

Regression labels need to be continuous and accurate. If your labels are noisy (user ratings are subjective, delivery times have measurement error) or naturally discrete (click/no-click), forcing regression can hurt performance. Classification is more robust to label noise because it only needs to get the category right, not the exact value.

One more real-world pattern worth knowing: many production systems use both. Netflix predicts whether you'll watch a show (classification) AND how much you'll enjoy it (regression) AND how long you'll watch (regression). Different predictions feed different parts of the system.

The home feed ranking uses predicted watch time. The "match score" percentage uses the enjoyment prediction. Content acquisition decisions use aggregate engagement classification models.