Spam detection is the system that sits between a sender and a recipient and decides whether a message is wanted. Gmail filters roughly 15 billion messages a day, Meta moderates billions of comments and DMs, and Twitter/X screens a steady stream of bot traffic. Rules catch the obvious stuff, but the long tail of phishing, coordinated campaigns, and evolving obfuscation is an ML problem.

Clarifying Questions

Candidate: "Which surface are we protecting? Email inbox, public comments, private DMs, and marketplace listings all have different latency and false-positive tolerances."

Interviewer: "Focus on email. The system has to decide before the message lands in the user's inbox."

Candidate: "What counts as spam here? Unsolicited promotional mail, phishing, scams, and automated bulk messaging all look different."

Interviewer: "All of those. The product team wants per-category labels so downstream teams can take different actions. Phishing gets blocked outright, promotional spam goes to a spam folder, scams get a warning banner."

Candidate: "What's the scale?"

Interviewer: "Around 10 billion messages a day, with peaks near 120K QPS. About 300 million active users. Roughly half the raw inbound traffic is spam before any filtering."

Candidate: "What's the latency budget per message?"

Interviewer: "The classifier has to return a decision under 100ms p99. SMTP servers can buffer, but delays above a few seconds start to cause timeouts for legitimate senders."

Candidate: "How do we feel about false positives versus false negatives?"

Interviewer: "False positives are much worse. A missed spam message is annoying. A legitimate business email that disappears can cost someone a job offer or a signed contract. Target less than 0.1% FP on legitimate mail, even if it means giving up some recall."

Candidate: "What labels can we rely on? User reports? Human review?"

Interviewer: "A mix. Users mark things as spam or 'not spam'. We run a honeypot of seed accounts that should never receive real mail. A small rater team adjudicates contested samples. Assume labels are noisy and delayed, sometimes by hours or days."

Requirements

Functional Requirements:

Non-Functional Requirements:

ML Problem Framing

The core task is multi-label text classification with heavy auxiliary signals. A message can carry more than one label, for example a phishing email that also uses promotional language. The output is a vector of calibrated probabilities per category, plus a top-level "is this spam" score used for the block/deliver decision.

Three properties make this different from a standard text classification problem.

First, class imbalance is real and asymmetric across categories. Ham dominates the post-filter stream, phishing is rare but catastrophic when missed. Naive sampling wrecks calibration. Training uses class-weighted loss and under-samples ham, then recalibrates on a held-out set that reflects production prevalence.

Second, concept drift is continuous. A phishing campaign that didn't exist yesterday can account for 5% of inbound volume by lunch. The model has to pick up new patterns in hours, not weeks. That rules out a monthly retrain cadence and forces a feedback loop that pushes fresh labels into training fast.

Third, the label signal is delayed and biased. A user reporting a message as spam hours after delivery gives you a clean positive. A user who never opens a message gives you nothing definitive, even though the message might be spam. This biases the training set toward messages that were salient enough to get a reaction.

Metrics

The system is a filter pipeline. Messages arrive at the ingress, flow through a series of progressively more expensive checks, and each stage can make a final decision or punt to the next one. Cheap checks catch the obvious stuff. Expensive checks handle the ambiguous cases.

The pre-filter is a straight lookup: known bad IPs, known bad content hashes, SPF/DKIM/DMARC validation, senders on the user's personal denylist. This drops 60-70% of inbound volume before any ML touches it. The linear model sees the remainder and makes fast decisions on clear cases. The content model, a transformer over the message body, runs only on the residual. The graph stage consumes sender-behavior features that are expensive to compute, so it often runs asynchronously and updates the decision post-delivery if needed.

Training runs on three separate loops, each tuned to a different kind of signal.

The daily loop trains on a large labeled corpus and produces a high-quality base model. The hourly loop fine-tunes on fresh user reports and honeypot signal, updating a lightweight adapter rather than the full model. The emergency loop kicks in when a campaign detector flags a sudden burst of similar content, pushing a hot-patched model in under an hour.

Scale Numbers

The back-of-the-envelope shapes most downstream design choices.

Data Flow

Inbound messages hit an edge SMTP cluster that strips envelope metadata, runs cryptographic validation, and forwards structured events to a streaming bus. From there, two paths diverge. The online path forks into the inference pipeline. The offline path fans the event out to a data warehouse, a feature pipeline, and a labeling pipeline that collects user reports and honeypot hits.

The online path has to make a decision in milliseconds. The offline path can take hours or days, because it feeds training rather than serving. The two paths share the event bus, which means training data reflects exactly what the serving system saw. That matters for debugging. When a specific message gets misclassified in production, the training pipeline has access to the same feature values that serving used, not a recomputed version that might differ subtly.

Data Sources

Labels

Labels come from four streams, each with a different precision-coverage trade-off. User "report spam" clicks are high-volume but noisy. Users sometimes report legitimate mail they don't want anymore, like a forgotten newsletter. Honeypot hits are rare but clean. If a message lands in a seed account, it's spam with very high confidence. Human raters handle the contested middle, labeling a sampled set for training and evaluation. Implicit signals like "deleted without opening" are soft negatives at best, since plenty of ham also gets ignored.

Label delay is a real problem. A phishing campaign can run for three hours before enough users report it to train an emergency model. That's three hours where the campaign is active and the detector is blind. Honeypot seeding shortens this window because attackers often scrape leaked address lists that contain seed accounts. The first honeypot hit of a new pattern is often the earliest signal available.

The other subtlety is that labels aren't symmetric. A "report spam" click is a strong positive. A "not spam" click on a quarantined message is a strong negative. But the absence of either click is ambiguous. A user who opens a message and does nothing might have decided it's fine, or might not have noticed it was spam. The training pipeline treats unreported messages as weak signal, weighted down compared to explicit actions.

Feature Engineering

Features break into five buckets. Content features describe the message itself. Sender features describe the sending entity. Recipient-relation features describe the history between sender and recipient. Graph features describe the sender's place in the broader send network. Temporal features capture bursts and seasonality.

URL features deserve their own treatment because URLs carry most of the phishing signal. Attackers use shortened URLs, redirect chains, homoglyph domains, and freshly registered domains to hide destinations. The feature pipeline expands short links, resolves redirect chains up to a depth limit, and checks the final destination against threat feeds.

Sender reputation is aggregated rolling stats keyed on the sending entity. The aggregation pipeline runs every five minutes for hot keys and hourly for the long tail.

Sample bias shows up in two places. First, the pre-filter drops the easy spam before training even sees it, so the model trains on the harder residual. Second, honeypot addresses never receive ham, so features learned from the honeypot stream can be systematically different from features of real users. The training pipeline handles both by maintaining a separate "pre-filter" training stream and by stratified sampling of real users for ham examples.

A common failure mode in feature engineering is leakage between training and serving. A feature like "sender complaint rate 7d" has to be computed as of the message arrival time, not using future complaints. If the training pipeline reads the feature store as it is today and joins it against messages from last week, the features contain information from the future. The model then looks great offline and collapses in production. Point-in-time correctness is enforced by the feature store: every feature lookup carries a timestamp, and the store returns the value that was live at that moment.

Baseline Approach

A logistic regression on token n-grams plus handcrafted sender features gets you further than you'd think. It's fast, interpretable, and its weights are easy to audit when an appeal comes in. For a decade, production spam filters were essentially this with better feature engineering.

The features are token unigrams and bigrams from subject and body, binary flags for a curated list of spam tokens, URL features, sender reputation features, and a small set of interaction terms like "new domain AND contains login URL". The model uses L1 regularization to keep the weight vector sparse, which makes scoring fast and lets a human read the top contributing features for any decision.

The ceiling on this approach is paraphrase. Spammers rotate wording, swap synonyms, and pad with benign text. A model that only sees bag-of-tokens misses that two messages with different surface forms are saying the same thing. It also misses cross-sender patterns. If the same campaign is sent from a hundred compromised accounts, each with its own sending domain, each message looks unusual-but-plausible in isolation. The model needs to see the pattern across senders.

Advanced Approach

Production systems layer a cascade. Each stage is tuned to a different budget.

Stage 1 is the same logistic regression described above, running in under a millisecond. It produces early-exit decisions for the clear cases on both ends of the score distribution, which covers roughly 80% of post-pre-filter traffic.

Stage 2 is a transformer encoder fine-tuned on message bodies, producing a contextual representation that feeds a classification head. The model sees subject, body, and selected metadata as a single sequence. Distillation from a larger teacher model keeps inference under 25ms on GPU batches. The encoder is warm-cached per language, because a message in Portuguese and a message in English hit different expert heads.

Stage 3 runs a graph neural network over the sender's local subgraph: other senders that share IP blocks, domains, content hashes, or recent recipients. It outputs a "coordinated campaign" signal that the linear and content models can't see on their own. This stage is often asynchronous. The decision service returns a preliminary verdict from stages 1 and 2, and the graph signal can reverse the decision within minutes if the sender's subgraph lights up.

The tradeoff between this cascade and a single monolithic model is cost. A single transformer large enough to match cascade recall would have to run on every message, which at 120K QPS and 25ms per inference is a lot of GPU. The cascade runs the expensive model only on the 15-20% of messages that stage 1 couldn't confidently resolve, cutting serving cost by roughly 5x at similar recall.

Training

The daily training job trains the full cascade end-to-end. Stage 1 trains on the full labeled pool with L1-regularized logistic regression. Stage 2 fine-tunes a base encoder on a balanced sample. Stage 3 trains on sender subgraphs extracted from the last 90 days of send activity.

Class weighting handles imbalance. With ham outnumbering confirmed spam 20 to 1 in the labeled pool, the loss is weighted so the gradient from a spam example counts 10-20x more. Hard-negative mining adds another lift. The pipeline sweeps the false-positive set from the previous model's predictions on held-out ham and uses those as high-weight negative examples in the next round. This sharpens the decision boundary where it matters.

The hourly incremental loop fine-tunes a lightweight adapter, typically a LoRA-style module on the transformer, using the last few hours of user reports and honeypot hits. It doesn't touch the base model, so rollback is a matter of swapping the adapter. If the incremental update regresses calibration or FP rate, the adapter gets reverted in under a minute.

Emergency retraining kicks in when the campaign detector flags a sudden cluster. A content-similarity job running over the last hour of inbound mail identifies bursts of near-duplicate messages. If the cluster is unlabeled and lands in honeypots, it triggers an emergency retrain focused on that cluster.

The emergency path has guardrails. Even in a rush, a hot-patched model has to clear a short offline evaluation that checks for FP regression on a small ham holdout. Without that check, an attacker could craft a campaign that looks enough like legitimate mail to poison an emergency retrain. The whole point of the cascade architecture is that adversaries can't easily steer the system into a bad place, and the emergency loop is the part most vulnerable to that, so it gets the tightest review.

Inference Pipeline

A message entering the pipeline hits a fan-out of feature lookups first, then the cascade, then a decision service that applies policy on top of the raw model scores.

The feature orchestrator fires lookups in parallel. Redis hosts the hot set of sender statistics, keyed by authenticated sender ID, domain, and IP. The offline feature store serves user preference vectors and per-tenant rules. The threat feed cache is a small local cache refreshed every few minutes from the main feed service.

The decision service layers policy on top of the raw scores. Enterprise customers can set a per-tenant sensitivity level. Individual users can allow-list a sender. Regulatory requirements in some regions require logging before blocking. The policy engine applies these rules after the model, so policy changes don't require model retraining.

Batch vs Real-Time

The key separation is between features that describe the message (real-time) and features that describe the sender's behavior over time (near-real-time or batch). A new sender that just started blasting spam has no reputation features yet, so the model has to rely on content signals for the first few minutes.

Latency Budget

Stage 3 graph features are kept off the synchronous path on purpose. Waiting for graph traversal inside the request would blow the latency budget. Instead, the decision can be revised asynchronously once the graph signal arrives, which is fine for email because delivery happens into folders that can be moved after the fact.

Graceful degradation matters more here than in most ML systems because the failure modes are both visible and bidirectional. If the feature store is down and the system falls back to "deliver everything", spam floods users. If the model service is down and the system falls back to "quarantine everything", legitimate mail disappears. The chosen fallback is to preserve the pre-filter and stage-1 linear model on cached weights, skip stages 2 and 3, and raise the operating threshold. This is conservative on FPs at the cost of lower recall, which matches the FP tolerance profile. An alert fires on any extended fallback so the service owner knows the system is running with diminished capability.

Adversarial Robustness

Spammers are not a stationary distribution. They probe the filter, learn what gets through, and adjust. This is the deep dive that shapes more of the system than any other.

Common obfuscation techniques include Unicode homoglyphs where a Latin 'a' gets replaced with a Cyrillic 'а', zero-width characters inserted to break token matching, image-only content to dodge text classifiers, URL shorteners and redirect chains to hide malicious destinations, base64 or HTML-entity encoded payloads, and content hidden in MIME parts that clients render but naive parsers skip.

Defenses are layered. Text normalization maps homoglyphs to a canonical form before tokenization, strips zero-width characters, and decodes HTML entities. Image attachments flow through an OCR pipeline that extracts text, and the OCR output feeds the same content model. URL expansion follows redirect chains up to a depth of five hops and classifies the final destination. MIME parsers are tuned to match client rendering behavior so the model sees what the user actually sees.

The harder defense is against adversaries who tune their content specifically to the model. Attackers with enough volume can treat the filter as an oracle: send variations, see which land in spam versus inbox, iterate. The countermeasure is a rate limit on the signal that adversaries can extract. The decision service adds noise to feedback when it's returned to senders (for instance, in NDR messages) and randomizes decisions near the threshold boundary. A small fraction of stage-1 decisions get elevated to stage 2 even when stage 1 is confident, which makes the oracle noisy.

Canary traps help catch adversaries early. A small pool of addresses in the honeypot receive different filtering treatments. If an attacker's volume drops sharply only on the treatment that blocked them, the filter's decisions are leaking. That's a signal to rotate thresholds and retrain.

False Positive Cost and Threshold Tuning

False positives aren't uniform. Blocking a promotional email the user didn't really want is a small cost. Blocking an order confirmation from a customer's bank is a much bigger cost. Threshold tuning has to reflect that.

The system uses per-surface thresholds. Phishing gets the strictest threshold because even a small number of delivered phishing messages causes real harm. Promotional spam gets a looser threshold because the failure mode is a missed coupon. The decision service maps category scores to actions through a policy table that's separate from the model.

Per-user thresholds also matter. A new account with no history is more likely to see FPs because the model has less personal context. A long-time user with a rich history of prior senders and clear folder preferences can tolerate a more aggressive threshold because the prior-contact signal is strong. The thresholding layer shifts based on user maturity and activity.

Quarantine versus hard-block is the other axis. A hard block returns a 5xx to the sending server, which bounces. Quarantine places the message in a spam folder the user can check. Hard blocks should be reserved for cases where false positives are recoverable (because the sender sees the bounce and can retry or contact support) and where delivery would cause outsized harm (phishing with credential theft links). Everything else gets quarantined.

The appeals path closes the loop. A user who marks a quarantined message as "not spam" updates their personal allow-list and also emits a training signal. An enterprise admin can override system-wide decisions for their tenant. Appeal reversal rate is a first-class metric, and a rising reversal rate is the fastest signal that a new model has drifted out of calibration.

The routing decision uses raw scores, per-category thresholds, and user-level overrides. A user who consistently rescues newsletters from the promotions tab teaches the personal rules engine to lower the threshold for that specific sender. This layered approach keeps policy changes cheap: adjusting a threshold or adding a per-tenant rule doesn't require retraining.

Graph Signals and Spam Rings

Content classifiers can be fooled. Coordinated campaigns are harder to hide because the pattern of who sends to whom leaves a structural signature.

A spam ring typically shows up as a cluster of senders that share one or more of: IP blocks, autonomous system numbers, sending infrastructure (same MTA fingerprint), content similarity (near-duplicate bodies with cosmetic variation), or recipient overlap (same list of addresses hit by many senders in a short window).

The graph pipeline builds a weighted sender-sender graph updated nightly. Edges are weighted by how much the two senders share. A connected-component analysis pulls out candidate rings. A graph neural network trained on labeled rings produces a "ring-likelihood" score per sender that feeds the cascade.

The diagram shows four senders linked through shared content hashes and a common recipient cluster. A single sender in that set looks plausible in isolation. The cluster, hitting the same recipients with near-identical content from a tight IP range, does not.

Content hashing uses SimHash or MinHash rather than exact hashes because spammers perturb content with random padding. SimHash produces a locality-sensitive fingerprint where small edits produce small Hamming-distance differences. Two messages with a SimHash Hamming distance under 5 on a 64-bit fingerprint are near-duplicates for practical purposes.

The tradeoff with graph signals is freshness. A brand new sending infrastructure has no graph history, so graph features are uninformative for the first hours. Content and sender-behavior features have to carry the first burst of a new campaign until the graph fills in.

Label Feedback Loop and Concept Drift

Spam evolves faster than most ML problems. A classifier that works today can miss 30% of the next campaign, and the next campaign might launch tomorrow. The feedback loop is the part of the system that has to keep up.

The loop has three streams feeding it. User reports are high-volume but noisy and biased. Users report things they find annoying, not only things that are spam in the strict sense. Honeypot hits are low-volume but clean. Human raters are lowest volume and highest quality, and handle cases where the user-report signal disagrees with the honeypot signal.

Label fusion combines signals using a simple precedence. Honeypot and rater labels override user labels when they conflict. Multiple user reports for the same message carry more weight than one. Reports against a sender the user has previously corresponded with are treated with suspicion, since a legitimate sender getting marked as spam is often a user who just unsubscribed mentally.

Selection bias is the hardest part. The training pool is biased toward messages users actually saw. Messages that were blocked outright never get reported, so the model trains on what the previous model couldn't confidently block. The pipeline corrects for this with a small holdout where a random sample of predicted-spam messages is delivered anyway, generating ground truth for the high-confidence region. This is a direct cost in FPs, but without it the model drifts into a bubble.

Shadow models address deploy risk. Any new model runs in shadow mode for a few hours before taking any traffic. Shadow mode scores every request but never changes the decision. The offline comparison between shadow and live predictions flags whether the new model is going to regress FP rate before users feel it. A staged rollout follows: 1%, 5%, 25%, 100% over 24-48 hours with automated rollback on any guardrail breach.

Offline Evaluation

Offline evaluation uses a temporal split. Random splits over time leak future information into training, which matters a lot here because attacker behavior changes week to week. The split holds out the last 7-14 days of labeled data and evaluates on that.

Key offline metrics: PR-AUC on the held-out window for each category, precision at the operating FP rate (typically 0.1%), recall on messages first seen less than 24 hours before the label, and calibration error across score buckets. A model that improves PR-AUC but loses calibration is not a good trade, because downstream surfaces rely on the probabilistic interpretation of scores to set their own thresholds.

Backtests against specific campaigns add qualitative signal. The evaluation pipeline keeps a library of known past campaigns and measures how fast the candidate model would have caught each one given the same label feedback loop. If the candidate is slower than the current production model on historical campaigns, something's wrong even if aggregate metrics look good.

Online Evaluation

Online evaluation runs A/B tests with a twist. Standard A/B splits users. For spam, splitting users biases the treatment's label feedback loop because half the user reports go to one model and half to the other. The system uses a dual-delivery design instead. A small fraction of the message stream gets scored by both models. When scores disagree, both decisions are logged but only one takes effect. User-report feedback on the delivered outcome updates labels for both models.

Primary online metrics: user spam-report rate per thousand delivered messages, FP complaint rate per thousand quarantined messages, time-to-catch on new campaigns. Guardrails: per-cohort FP rate (especially for small businesses, new domains, non-English senders), appeal reversal rate, and end-to-end latency p99.

An online test runs for at least 7 days to span weekly seasonality. Campaign cycles often last a few days, so a 3-day test can miss the regression that a new campaign would trigger.

One practical challenge with dual-delivery is that user reports on disagreement cases have to be attributed carefully. If model A said spam and model B said ham, and the message got delivered, a later "report spam" click credits both models with a label, but only model A got the decision right in real time. The evaluation pipeline tracks both the score agreement and the eventual label to separate "the new model is better" from "the new model just got lucky on this cohort".

Monitoring

Production monitoring runs at a faster cadence than evaluation. The system watches score-distribution drift minute-by-minute using population stability index (PSI) between the current hour and a rolling baseline. Feature-level drift runs per-feature and flags when a feature's distribution shifts significantly. A sudden jump in the "domain age" feature median, for example, often means an attacker registered a new batch of domains.

Honeypot catch rate is the fastest early warning. Honeypot addresses should never receive ham, so anything less than near-100% catch rate on honeypot traffic means the model is missing real spam. A drop from 99% to 90% is a five-alarm fire.

Per-cohort monitoring catches fairness regressions that aggregate metrics hide. A new model might improve overall precision while regressing on small-business domains or non-English senders. The dashboard breaks FP rate by sender country, sender domain age bucket, language, and authentication status. A regression on any bucket triggers a review even if top-line metrics look clean. Fairness isn't only an ethical concern here, it's a product concern: a filter that systematically junks legitimate mail from Brazilian small businesses will lose those users, and that loss won't show up in global metrics for weeks.

The iteration cadence combines the daily full retrain, the hourly incremental adapter, and the emergency retrain hook. Daily retrains ship behind shadow mode and a staged rollout. Hourly adapters ship with a fast-revert, since they're small deltas. Emergency retrains bypass the staged rollout only when the campaign detector fires on a cluster that's both large and landing in honeypots, which is rare but time-critical.