{"title":"Design an ETA Prediction System","description":"","content":"An ETA prediction system estimates how long a journey will take from now until arrival, and serves that estimate to riders, couriers, dispatchers, and downstream systems like pricing and routing. What makes it an ML problem rather than a routing problem is that the physical shortest path gives you a distance, but the time depends on dozens of variables the graph alone doesn't see: traffic, weather, time of day, driver behavior, restaurant prep speed, whether the courier is still picking up from the restaurant or already en route to the drop-off. A good design is a stacked regressor that predicts each leg of the journey separately, quantifies uncertainty rather than returning a single number, and refines its estimate as fresh GPS pings arrive.\n\n---\n\n# 1. Problem Formulation\n\n### Clarifying Questions\n\nETA systems look deceptively similar across products but differ in what the prediction represents, which edges of the journey the ML model owns, and how the number is consumed. Pinning these down first changes almost every later design decision.\n\n**Candidate:** \"What product surface are we building ETA for? Rideshare, food delivery, package delivery, or something like Google Maps directions?\"\n\n**Interviewer:** \"Food delivery on a platform that handles about 10 million orders a day across 30 countries. Think DoorDash or Uber Eats shape. The user-facing ETA shown at order time is the full journey from order placement to drop-off at the customer, not just the drive time.\"\n\n**Candidate:** \"That's a multi-leg journey: courier travels to the restaurant, waits for food, then drives to the customer. Are we predicting the total end-to-end time, or each leg separately?\"\n\n**Interviewer:** \"That's exactly the design question. Product needs a single number to show the customer, but internally you can decompose however you want. The merchant-prep time varies a lot by restaurant, by time of day, and by order size, so treating it as a black box inside one model probably won't work.\"\n\n**Candidate:** \"Where is the ETA used? Is it shown only to the customer, or do dispatching and pricing depend on it too?\"\n\n**Interviewer:** \"Three consumers. The customer sees the ETA at order time and gets live updates during the trip. Dispatch uses the per-leg estimates to assign couriers, because a courier who can reach the restaurant in 3 minutes is a better match than one who can reach it in 8. Pricing uses the expected delivery duration for dynamic fees and surge logic. Different consumers care about different things: the customer wants accuracy and calibrated uncertainty, dispatch wants fast refreshes at low latency, pricing tolerates more latency but wants unbiased estimates.\"\n\n**Candidate:** \"What's the latency budget? The customer-facing ETA at order time is probably more forgiving than dispatch.\"\n\n**Interviewer:** \"Dispatch runs it at 50,000 requests per second during peak meals with a p99 budget of 150ms because it scores thousands of courier-order pairs per dispatch cycle. Customer-facing prediction at order time can take up to 500ms. Mid-trip live updates run every 10 seconds and share the dispatch latency budget.\"\n\n**Candidate:** \"How is accuracy measured? MAE on delivery time, or something business-facing?\"\n\n**Interviewer:** \"Both. Offline we use MAE and quantile loss. The online metric that matters most is the fraction of deliveries where actual time exceeds the customer-facing ETA by more than 5 minutes, because that's the complaint trigger. Under-promising pads the ETA and depresses conversion; over-promising breaks trust. The sweet spot is a slight pessimistic bias, and the product decides how much.\"\n\n**Candidate:** \"What's the feedback timeline? When do we know the actual time?\"\n\n**Interviewer:** \"The drop-off confirmation happens in real time, so labels arrive within seconds of the trip ending. Unlike fraud, there's no 90-day wait. The main label quality issue is that some drops get confirmed by the courier on arrival while others get auto-confirmed after the fact, so there's a small amount of label noise on the trailing tens of seconds.\"\n\n**Candidate:** \"What about extreme conditions? Weather events, stadium lets out, a road closes because of an accident.\"\n\n**Interviewer:** \"All of them happen daily somewhere. The design needs to handle them without tripping over itself. Baseline models often blow up on rain days because traffic patterns change discontinuously.\"\n\n### Requirements\n\n**Functional Requirements:**\n\n- Predict expected delivery time in seconds, plus a calibrated uncertainty interval (for example p10 to p90), for every courier-order pair considered by dispatch.\n- Decompose the prediction into named legs: courier-to-restaurant drive time, restaurant dwell time (order preparation plus hand-off), and restaurant-to-customer drive time.\n- Refresh the estimate in near-real time as the trip progresses, triggered by courier GPS pings, restaurant confirmations, and traffic updates.\n- Expose a single rolled-up ETA to the customer and a per-leg breakdown to dispatch and pricing.\n- Return an explicit signal when the model is operating outside its training distribution (new geography, unusual weather, novel road network).\n\n**Non-Functional Requirements:**\n\n- P99 latency under 150ms for dispatch-path scoring, under 500ms for order-time prediction.\n- Availability of 99.95% for the scoring service; dispatch stalls if the service is down.\n- Calibrated probabilistic outputs: the empirical coverage of the 80% interval should be within 2 percentage points of 80% across all major slices.\n- Graceful degradation: if the ML scorer is unavailable, a fallback based on historical segment speed tables and merchant prep priors maintains service at reduced accuracy.\n- Fairness across neighborhoods: the model cannot systematically under-estimate delivery time in lower-income or rural zip codes in a way that biases dispatch against those regions.\n\n### ML Problem Framing\n\nThe natural framing is per-leg regression with distributional outputs. Let a delivery consist of three legs: pickup drive (courier current location to restaurant), dwell (arrival at restaurant to departure with order), and drop-off drive (restaurant to customer). For each leg, the model predicts not a single number but a distribution over time-to-complete, typically parameterized by a handful of quantiles or the parameters of a mixture density. The total delivery ETA is the sum of the three leg predictions, with uncertainty propagated through the sum.\n\nTwo design choices in the framing matter more than the choice of architecture. First, training one model per leg is cleaner than training an end-to-end model on total time. The legs have different feature dependencies (dwell time depends almost entirely on restaurant and order attributes, drive time depends on road network and traffic), and a shared model would either split its capacity poorly or require complex gating. Per-leg models also allow independent iteration: improving dwell-time prediction doesn't require retraining the drive-time model.\n\nSecond, the loss function has to match the asymmetric cost of errors. Under-estimating by 5 minutes costs a customer complaint, over-estimating by 5 minutes costs conversion. Quantile loss at multiple quantiles (p10, p50, p90) captures the distribution, and the action is to show the customer a point between p50 and p70 depending on product preference. A plain MSE loss is the wrong objective because it optimizes a symmetric penalty and ignores the business asymmetry.\n\n### Metrics\n\n\n| Type | Metric | Why It Matters |\n|---|---|---|\n| Offline | MAE on delivery time | Primary point-prediction accuracy |\n| Offline | Quantile loss at p50, p70, p90 | Measures distributional fit, used to pick the served quantile |\n| Offline | Bias (mean residual) | Flags systematic over- or under-prediction |\n| Offline | Coverage of 80% interval | Empirical fraction inside [p10, p90]; target 80% +/- 2 pts |\n| Offline | Leg-wise MAE | Isolated accuracy for pickup, dwell, drop-off |\n| Offline | Slice MAE | Broken out by city, time-of-day, weather condition, distance band |\n| Online | Fraction of \"lateness events\" | Deliveries where actual time exceeds customer ETA by >5 min |\n| Online | ETA over-promise rate | Fraction where customer ETA is shorter than actual |\n| Online | Dispatch accuracy | Percentage of dispatches where the chosen courier arrived within the predicted window |\n| Online | Live-update correction | Average size of mid-trip ETA updates; large corrections signal poor initial predictions |\n| Guardrail | Customer complaint rate | Support tickets mentioning ETA per million orders |\n| Guardrail | Conversion rate at checkout | Catches over-padded ETAs that depress orders |\n| Guardrail | Subgroup lateness ratio | Lateness rate in the worst-serviced city versus the best; catches fairness regressions |\n\n\nThe lateness event rate gets singled out because it's the metric that connects directly to customer satisfaction and also has the cleanest interpretation: a 12% lateness rate means 12% of deliveries in the last week ran more than 5 minutes late relative to the ETA we promised. It's the metric every review comment, support escalation, and executive dashboard eventually maps back to.\n\n---\n\n# 2. System Architecture\n\nA delivery's ETA is computed many times over its lifetime. Once at order time when the customer sees the initial estimate, thousands of times during dispatch as the system evaluates candidate couriers, and then every few seconds during the trip as fresh GPS pings and state transitions arrive. The architecture has to serve all three with different latency and freshness characteristics while sharing features and models.\n\n\n```mermaid\nflowchart LR\n REQ[ETA
Request]:::primary --> ROUTE[Routing
Service]:::teal\n ROUTE --> FEAT[Feature
Fetch]:::teal\n FEAT --> LEG[Leg-wise
Predictor]:::primary\n LEG --> AGG[Aggregator
+ Quantile
Selector]:::orange\n AGG --> RESP[ETA +
Interval]:::green\n AGG -.log.-> LOG[Decision +
Feature Log]:::teal\n GPS[Courier GPS
Ping]:::primary -.trigger.-> LEG\n STATE[Order State
Transition]:::primary -.trigger.-> LEG\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\nThe routing service produces the physical route between two points as a sequence of road segments along with raw distance and the graph's baseline travel-time estimate assuming no traffic. Everything after that is ML. Feature fetch pulls segment-level traffic, restaurant-level prep priors, courier-level speed history, and global context (weather, day-of-week, lunch peak flag). The leg-wise predictor runs pickup-drive, dwell, and drop-off-drive models in parallel and returns three distributions. The aggregator sums them, applies the product's quantile preference (typically p65 for the customer-facing number), and returns a final ETA with an interval.\n\nThree asynchronous loops keep the inputs fresh without blocking any single request.\n\n\n```mermaid\nflowchart TD\n subgraph Traffic[Traffic Loop - seconds to minutes]\n GPS[GPS Pings
from fleet]:::primary --> SEG[Segment
Speed Aggregator]:::teal\n SEG --> TSTORE[Traffic
Feature Store]:::orange\n INC[Incident
Feed]:::primary --> TSTORE\n end\n\n subgraph Prep[Prep Time Loop - hourly]\n ORDS[Order
History]:::primary --> PAGG[Merchant
Prep Aggregator]:::teal\n PAGG --> PSTORE[Merchant
Feature Store]:::orange\n end\n\n subgraph Train[Training Loop - daily]\n LOG[Feature +
Outcome Logs]:::primary --> JOIN[Label Join
+ Leg Split]:::teal\n JOIN --> TRAIN[Quantile GBDT
+ Sequence Model
Training]:::orange\n TRAIN --> CKPT[Model
Checkpoint]:::green\n end\n\n TSTORE -.read by.-> SRV[Scorer Fleet]:::primary\n PSTORE -.read by.-> SRV\n CKPT -.served by.-> SRV\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\nThe traffic loop is the fastest: every active courier's GPS pings flow into a streaming aggregator that computes segment-level speeds on rolling 2-minute windows, and the result lands in an online feature store keyed by segment ID. The prep-time loop updates per-merchant prep priors hourly, because restaurant speed doesn't change second by second but does vary across lunch versus dinner, weekdays versus weekends. The training loop retrains daily on joined feature and outcome logs from the last 30 days.\n\nThe state transitions matter for when to re-score. A single delivery might be scored 200 to 400 times across its lifetime: once at order submission, a few hundred times during courier dispatch, and then every 10 to 30 seconds from pickup through drop-off. Each of those scorings uses different features because more is known as time progresses, and the model needs to handle the changing feature availability gracefully.\n\n\n| Dimension | Number | Implication |\n|---|---|---|\n| Order volume | 10M/day, peak 500K/hour | Dispatch path sees 50K req/s during meal peaks |\n| Active trips at peak | ~800K concurrent | Live-update loop runs ~80K re-scorings per second |\n| Latency budget (dispatch) | 150ms p99 | Feature fetch dominates; parallel per-leg scoring |\n| Latency budget (order time) | 500ms p99 | Allows heavier routing + model ensemble |\n| Road network size (global) | ~500M segments | GNN must use neighborhood sampling, not full-graph |\n| Merchants | ~2M active globally | Per-merchant prep priors with prior smoothing for long tail |\n| Label arrival | Seconds after drop-off | Fast feedback enables near-continuous retraining |\n| Retraining cadence | Daily full, hourly incremental | Balances freshness with deployment overhead |\n\n\n---\n\n# 3. Data\n\n### Sources\n\nThe signal for ETA comes from five layers: the physical road network, real-time state of that network (traffic, incidents), the merchant side of the transaction, the courier side, and the global context that shifts all of them together (weather, events, time of day).\n\n\n| Source | Content | Update Cadence |\n|---|---|---|\n| Road network graph | Segments, intersections, road class, speed limits, turn restrictions | Weekly |\n| Courier GPS stream | Position, speed, heading, timestamp from every active courier | ~2 Hz per courier |\n| Segment speed aggregator | Rolling-window speed per segment from aggregated GPS | 1-2 minutes |\n| Traffic incident feed | Accidents, construction, closures from third-party + operations ops | Real-time |\n| Merchant order history | Per-merchant prep times, hand-off times, order complexity | Hourly batch |\n| Order state stream | Placed, accepted, prepared, picked up, delivered events | Real-time |\n| Courier profile | Vehicle type (car, bike, scooter, walker), speed priors, tenure | Daily |\n| Weather feed | Precipitation, temperature, wind at 1km grid resolution | 15 minutes |\n| Event calendar | Sports games, concerts, parades, rush hour variations | Daily |\n| Historical delivery logs | Feature snapshots + actual durations for every past delivery | Continuous |\n\n\nThe courier GPS stream is the backbone. Without it, segment-level speeds would degrade to public traffic feeds that lag by 10 to 30 minutes and miss the micro-patterns that show up in delivery-specific corridors (shortcut alleys, pedestrian zones bikes can cut through). The pings are aggregated up to segment-level speed with privacy-preserving windowing: no individual courier's position is exposed downstream, and any segment with fewer than 3 distinct couriers in the window falls back to historical priors rather than revealing per-courier data.\n\nMerchant order history is the second most important input and the most under-appreciated. A pizza place that takes 22 minutes to prepare an order on Friday nights but 9 minutes on Tuesday afternoons is the single biggest variance contributor in a city with predictable traffic. The prep-time aggregator rolls up order history with explicit breakouts by day-of-week, hour-of-day, order size, and menu category, smoothed against a global prior so new merchants with thin history don't blow up the tail.\n\n### Feature Engineering\n\nFeatures group into six categories based on the layer of signal they come from and their freshness characteristics. The split matters because different categories have different failure modes and different update cadences.\n\n**Route features** describe the physical path. Total distance (great-circle and driving), number of segments, count of turns, count of traffic signals, max road-class level along the route, presence of limited-access segments (highways, bridges, tunnels). These come directly from the routing service and are effectively free to compute per request.\n\n**Traffic features** describe the current state of the road network. Per-segment speed relative to historical baseline, aggregated along the route to produce a weighted \"traffic coefficient,\" incident flags on segments on the route, speed-of-traffic deceleration (a segment that was clear 5 minutes ago and is now slow), and distance to the nearest active incident. These are the most volatile features and have the highest correlation with same-day accuracy.\n\n**Temporal features** encode when the trip happens. Hour-of-day bucketed into 30-minute bins, day-of-week, whether it's a meal peak window (lunch 11:30-13:30, dinner 18:00-20:30), holiday flag, school-in-session flag. Temporal features are cheap, stable, and carry strong signal because congestion patterns are deeply periodic.\n\n**Merchant features** capture the dwell-time layer. Historical median prep time, tail percentiles (p90, p95), prep-time volatility, current order queue depth at the merchant (how many pending orders they're already preparing), recent confirm-time accuracy (how close their estimates track reality), and order complexity features (item count, custom instructions, menu category).\n\n**Courier features** describe the entity traveling. Vehicle type, historical median speed on equivalent road types, tenure bucket, time since last delivery completion (warm versus cold), current location relative to route start, active batching state (is this courier already carrying another order?).\n\n**Weather and event features** are the global context. Precipitation type and intensity, temperature, wind speed, visibility, and flags for ongoing events (stadium events within 2km of the route). Weather features are one-dimensional but high-leverage: a rainy Friday at 6pm in Manhattan has distributional properties unlike any other day, and omitting rain-hour interactions produces a model that fails predictably on the worst traffic days.\n\n\n| Feature | Type | Source | Freshness | Description |\n|---|---|---|---|---|\n| route_distance_m | Float | Routing service | Real-time | Driving distance along the chosen route |\n| route_signal_count | Int | Routing service | Real-time | Number of traffic signals on the route |\n| route_turn_count | Int | Routing service | Real-time | Count of turns weighted by turn complexity |\n| segment_speed_ratio | Float | Traffic feature store | 1-2 min | Avg current speed / historical speed, route-weighted |\n| incident_distance_m | Float | Derived | Real-time | Distance to nearest active incident on the route |\n| hour_of_day_bin | Int | Derived | Real-time | 30-minute bucket of local time |\n| day_of_week | Int | Derived | Real-time | 0-6, with holiday override |\n| meal_peak_flag | Bool | Derived | Real-time | True during lunch or dinner peak windows |\n| merchant_prep_p50 | Float | Merchant store | Hourly | Median prep time for this merchant, same hour-of-day |\n| merchant_prep_p90 | Float | Merchant store | Hourly | 90th percentile prep time |\n| merchant_queue_depth | Int | Order state | Real-time | Pending orders already in this merchant's queue |\n| order_item_count | Int | Order payload | Real-time | Number of items in the order |\n| courier_vehicle_type | Enum | Profile store | Daily | Car, bike, scooter, walking |\n| courier_speed_prior | Float | Profile store | Daily | Historical median speed for this courier on similar roads |\n| courier_batch_state | Enum | Dispatch | Real-time | Solo, batched with 1 other, batched with 2+ |\n| weather_precip_mm | Float | Weather feed | 15 min | Precipitation rate in mm/hour at route centroid |\n| temp_c | Float | Weather feed | 15 min | Temperature in Celsius |\n| event_within_2km | Bool | Event calendar | Hourly | True if a scheduled event is active within 2km of route |\n\n\nThe full feature set is closer to 150 to 250 features once you include interactions and segment-level vectors, but the table above captures the representative types. The segment-level speeds are themselves a variable-length vector because routes have variable segment counts; handling that in the model is a key design choice covered in the deep dives.\n\n### Labels and Fast Feedback\n\nUnlike classification problems where labels arrive slowly, ETA labels are fast: the drop-off confirmation fires within seconds of trip completion, and the actual elapsed time is just a subtraction. This is one of the reasons ETA systems can retrain aggressively, on timelines of hours rather than weeks.\n\nThere are still label quality issues to handle. Some drop-offs are confirmed by the courier on arrival (clean label), others are auto-confirmed after a timeout because the courier forgot to tap the button (label has up to a minute of spurious inflation). The training pipeline filters out auto-confirms with a noisy-label flag and either drops them or weights them down. Trips that get cancelled mid-flight become negative examples for cancellation models but are excluded from ETA training because they never completed.\n\nA subtler issue is that the distribution of actual durations is censored: late trips sometimes get cancelled, picked up by a second courier, or manually resolved, and those don't appear in the training data. Censoring biases models toward optimism because the worst outcomes are missing from the training set. The fix is to retain censored trips in the training data with a censoring indicator, treating the censoring time as a lower bound on the true duration, and using a censoring-aware loss like a Tobit regression or a modified quantile loss that treats censored examples appropriately for the upper quantiles.\n\n---\n\n# 4. Model\n\n### Baseline: GBDT with Quantile Loss\n\nThe baseline is a LightGBM regressor trained with quantile loss at three quantiles (p50, p70, p90) over the feature set above, producing three separate boosters whose outputs form the predicted distribution. Quantile loss is the pinball loss: for quantile `q`, the loss on residual `r = y - y_hat` is `q * max(r, 0) + (1 - q) * max(-r, 0)`, which pulls the prediction toward the `q`th percentile of the conditional distribution.\n\nTraining runs per leg: a pickup-drive model, a dwell model, and a drop-off-drive model, each with its own feature selection appropriate to that leg. The pickup-drive and drop-off-drive models share most route and traffic features; the dwell model uses merchant and order features with almost no routing input. Output quantiles are summed across legs to produce the total delivery distribution.\n\nThe baseline is not a toy. In a mature production system, a well-tuned per-leg quantile GBDT reaches within 5 to 10% of the best deep-learning architectures on aggregate MAE, trains in under an hour, serves in under 5ms per prediction, and is easy to monitor and debug. The gap between the baseline and the advanced model is concentrated in tail accuracy (p95 and beyond) and in slices where the feature engineering budget is the bottleneck. For many platforms, the GBDT is the production model for years before deep models earn their place.\n\n### Advanced: Leg-Wise Stack with Road-Network Encoder\n\nThe production architecture in mature platforms combines three model types: a road-network encoder that turns a route into a vector, a sequence model that integrates temporal dynamics, and a mixture density head that emits a distributional prediction. Each leg has its own stack, and the stacks share the road-network encoder weights.\n\n\n```mermaid\nflowchart LR\n ROUTE[Route
segments]:::primary --> GNN[Road-Network
GNN Encoder]:::orange\n CTX[Context
features]:::primary --> TRF[Transformer
over route
+ context]:::orange\n GNN --> TRF\n MERCH[Merchant +
Order features]:::teal --> PREP[Prep-Time
Sub-Model]:::orange\n TRF --> PICK[Pickup-Drive
Head]:::primary\n TRF --> DROP[Drop-off-Drive
Head]:::primary\n PREP --> DWELL[Dwell Head]:::primary\n PICK --> AGG[Sum + Variance
Propagation]:::orange\n DWELL --> AGG\n DROP --> AGG\n AGG --> OUT[p10 / p50 / p90
ETA Distribution]:::green\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\nThe **road-network GNN encoder** takes the route as a sequence of segment IDs plus each segment's static attributes (length, road class, speed limit) and real-time attributes (current speed, incident flag). A two-layer GraphSAGE aggregates neighborhood features to produce a per-segment embedding that captures local network structure. At scale (hundreds of millions of segments globally), the GNN uses neighborhood sampling just like the fraud-detection graph: at each layer, each segment attends to a sampled subset of its neighbors rather than the full graph. The embedding captures patterns like \"this segment connects two high-speed arterials\" or \"this segment has three traffic signals within 500m\" that per-segment features miss.\n\nThe **Transformer layer** takes the sequence of per-segment embeddings plus a pooled route context (total distance, turn count, temporal features) and produces a fixed-size route representation. The attention mechanism is where the model learns that the first few segments matter more than the later ones for near-term predictions (because traffic upstream of the courier is less relevant than traffic on the segment they're about to enter). A couple of transformer blocks is enough; deeper stacks start overfitting to frequent routes.\n\nThe **prep-time sub-model** is a separate small GBDT or MLP operating on merchant and order features. Dwell time has almost no dependency on the road network, so training it inside the main stack wastes capacity and dilutes gradients. Running it as a side model with independent features is cleaner and lets the merchant ops team iterate on it without retraining the big model.\n\nThe **mixture density head** on each leg emits parameters of a mixture of three Laplace or Gaussian distributions. Laplace is the more common choice because ETA residuals have heavier tails than Gaussian, and the Laplace likelihood is less punishing on rare large errors. At serving time, the quantiles are computed analytically from the mixture parameters, and the three leg distributions are convolved to get the total delivery distribution (or approximated by summing means and variances with a correction for correlation, which is good enough in practice).\n\n### Training\n\nTraining data is 30 days of completed deliveries joined to feature snapshots at the time of each scoring event. The joining is non-trivial: a single delivery has many scoring events (order time, dispatch, mid-trip updates), and for each event the feature values were whatever they were at that moment. Training on the order-time feature snapshot against the final duration is the canonical setup; training on mid-trip snapshots against the remaining duration is used for the live-update model, which is typically a separate head that shares the encoder.\n\nThe loss is a combination of quantile loss (for the quantile heads in the baseline or as an auxiliary loss for the advanced model) and the negative log-likelihood of the mixture density model (for the primary distributional head). A weighting factor balances the two so that training optimizes both the quantile-specific calibration and the full distributional fit.\n\nTemporal splits matter for the same reason they matter in fraud: random shuffling leaks future patterns into training. The convention is to train on weeks 1 through 4, validate on week 5, and test on the most recent complete week. A rolling walk-forward variant is used for final benchmark numbers.\n\nClass-imbalance issues are milder than in fraud because ETA isn't imbalanced in the positive-class sense, but the training distribution is imbalanced across contexts. Lunch-hour peaks are 4 to 6 times denser than 3am, rain-hours are rare events, and the worst-traffic cities generate a disproportionate share of hard examples. Upweighting rare-context examples (rain, meal peaks, extreme distance) brings their loss contribution in line with their operational importance, though it comes at some cost to aggregate MAE because easy examples are now relatively under-weighted.\n\nFeature store serving requires training-serving consistency: the feature values seen during training must match the values served at prediction time. The standard defense is a point-in-time join that replays the feature store's log to reconstruct what each feature's value was at each historical scoring event, rather than joining against the current value of each feature.\n\n---\n\n# 5. Serving\n\n### Inference Pipeline\n\nThe serving path at dispatch time looks like this.\n\n\n```mermaid\nflowchart LR\n REQ[Dispatch
Request
t=0ms]:::primary --> RT[Routing
Service
+20ms]:::teal\n RT --> FEAT[Parallel
Feature Fetch
+25ms]:::teal\n FEAT --> PICK[Pickup-Drive
Model
+15ms]:::primary\n FEAT --> DWELL[Dwell
Model
+5ms]:::primary\n FEAT --> DROP[Drop-off-Drive
Model
+15ms]:::primary\n PICK --> AGG[Aggregator
+3ms]:::orange\n DWELL --> AGG\n DROP --> AGG\n AGG --> RESP[ETA +
Interval
+2ms]:::green\n AGG -.log.-> LOG[Feature +
Prediction Log]:::teal\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\nRouting runs first because most features depend on the route. The routing service is CPU-intensive and often cached: for the same origin-destination pair within a short window, the cached route is reused unless a new incident appears on it. Cache hit rates at peak run around 35 to 50%, which cuts the average routing cost significantly.\n\nFeature fetch is the latency choke-point. The service issues parallel lookups to the segment-speed store (one lookup returns speeds for all segments on the route in a batched call), the merchant feature store, the courier profile cache, and the weather store. The request is assembled into a dense feature vector with a freshness bitmap indicating which features came from cache misses or stale entries, and the model has seen training examples with the same freshness patterns so it can implicitly downweight stale features.\n\nModel scoring runs the three leg models in parallel. The pickup-drive and drop-off-drive models share the GNN encoder on their respective routes, but because the two routes are different (different segments, different traffic state), the encoder runs twice. Batching across concurrent dispatch requests is how throughput scales: a single GPU can score 200 to 500 route predictions per forward pass if batching is done right, and at 50K req/s the dispatch path needs aggressive micro-batching to stay in budget.\n\n### Latency Budget\n\nThe 150ms p99 dispatch budget breaks down like this.\n\n\n| Stage | Budget | Notes |\n|---|---|---|\n| Network + parsing | 8ms | Gateway to ETA service RPC |\n| Routing service | 25ms | Shortest-path computation or cache lookup |\n| Feature fetch | 30ms | Parallel multi-get across 4 stores |\n| Pickup-drive model | 18ms | GNN + Transformer + head, GPU or CPU batched |\n| Dwell model | 5ms | Small MLP or GBDT |\n| Drop-off-drive model | 18ms | Parallel with pickup-drive |\n| Aggregator | 3ms | Distribution sum, quantile extraction |\n| Logging + response | 8ms | Async log emit, serialization |\n| Buffer | 35ms | Tail variance in feature fetch and routing |\n\n\nThe buffer is deliberately large because both routing and feature fetch are bimodal. Routing median is 8ms but cold-cache routes (new destinations, rerouted trips) can take 40ms. Feature fetch median is 12ms but fleet-wide Redis shards occasionally hit gc pauses that push individual requests to 60ms. A smaller buffer would make the SLA brittle under load.\n\n### Batch vs Real-Time\n\nThe dispatch-path scorer is hard real-time, so are the mid-trip refresh scorings. The segment-speed aggregator is streaming (seconds-to-minutes windows), and the merchant prep-time aggregator is hourly batch because merchant behavior doesn't drift faster than that. Model retraining is daily.\n\nThe interesting near-real-time component is the live-update scorer. During an active trip, the courier's GPS pings every 500ms to 2 seconds, but re-scoring ETA on every ping would be wasteful (traffic doesn't change that fast, and the customer only cares about updates of 30 seconds or more). The refresh policy is adaptive: re-score when the courier has traveled more than 100 meters, or when 30 seconds have passed, or when a major state transition fires (order ready at merchant, courier arrived at drop-off neighborhood). The resulting refresh rate is one scoring every 10 to 30 seconds per active trip, averaging to about 100K re-scorings per second across 800K concurrent trips.\n\n### Degraded Serving\n\nWhen the model service is unavailable or feature fetch starts timing out, the fallback path is a lookup-table approach: segment-speed tables by hour-of-day and day-of-week, plus merchant prep priors, combined with a simple additive model. The fallback is not retrained on the fly and is roughly a stripped-down version of the 2018-era pre-ML ETA system, which is actually good news because it means the fallback is well-understood and battle-tested. Accuracy in fallback mode runs 20 to 30% worse on MAE than the full model, which is painful but much better than failing open.\n\nLike fraud, the fallback is exercised weekly in a chaos-engineering drill. A single region's scorer is taken offline for 5 minutes, and the degraded-mode lateness rate, courier utilization, and customer complaint rate are measured against a baseline. Fallback paths that aren't exercised rot, and the first time a broken fallback shows up in a real outage is a bad day.\n\n---\n\n# 6. Deep Dives\n\n### 6.1 Route Representation: Segment-Based vs Graph-Based\n\nThe choice of how to represent the route to the model has outsized influence on both accuracy and infrastructure cost. The two main approaches are segment-based encoding and graph-based (GNN) encoding, and they sit at different points on a cost-accuracy curve.\n\nIn the **segment-based approach**, the route is flattened into a vector of hand-crafted features: total distance, count of high-class segments, count of signals, average speed along the route weighted by segment length, maximum current traffic slowdown, and so on. The GBDT eats this vector directly. Feature engineering is the development work, and each new feature is a code change plus a backfill. The approach is cheap (feature computation is linear in route length, and no GPU is needed) and interpretable (SHAP values show exactly which route attribute mattered most).\n\nIn the **graph-based approach**, the route is kept as a sequence of segment IDs, and the model runs a GNN over the road network to produce per-segment embeddings that get pooled into a route representation. The embedding captures structural information about each segment's context: how many high-class segments are nearby, how well-connected the segment is, whether it sits on a frequent shortcut route. These are hard to hand-engineer because they involve two-hop or three-hop reasoning about the graph.\n\n\n| Dimension | Segment-Based | Graph-Based |\n|---|---|---|\n| Feature engineering | Manual, per-feature code changes | Learned, from neighborhood aggregation |\n| Training cost | Low; CPU, hours | High; GPU, hours to days |\n| Serving cost | Low; CPU, sub-5ms | Higher; GPU or CPU-batched, 10-30ms |\n| Interpretability | High; per-feature SHAP | Lower; embeddings are opaque |\n| Accuracy on common routes | Similar | Similar to slight edge |\n| Accuracy on rare routes | Degrades with sparse training | Better generalization via graph structure |\n| Cold-start for new segments | Requires backfill | Embeddings fall back to neighborhood |\n\n\nThe honest answer is that the GNN's edge is concentrated in three situations: routes through under-represented geographies, novel segments that haven't been in training data, and long routes where two-hop context matters more than single-segment features. For the middle 80% of routes on well-travelled corridors, a GBDT with segment-based features matches or beats the GNN on MAE while running at a fraction of the cost.\n\n\n```mermaid\nflowchart LR\n subgraph Seg[Segment-Based]\n RTE1[Route]:::primary --> FEAT1[Hand-crafted
feature vector]:::teal\n FEAT1 --> GBDT[LightGBM]:::orange\n GBDT --> ETA1[ETA]:::green\n end\n\n subgraph Graph[Graph-Based]\n RTE2[Route]:::primary --> SAMPLE[Neighborhood
Sampler]:::teal\n SAMPLE --> MP[Message
Passing]:::orange\n MP --> POOL[Pool +
Transformer]:::orange\n POOL --> ETA2[ETA]:::green\n end\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\nMost mature platforms end up running both, not as a choice but as a stack. The GBDT handles the high-volume fast path, and the GNN feeds embeddings into the GBDT as additional features or scores a fraction of traffic where the segment-based features are known to be weak (new geographies, atypical routes). The hybrid captures the GNN's generalization benefit without paying its serving cost on every request.\n\n### 6.2 Uncertainty Quantification\n\nShowing the customer a single number for ETA is a product choice that hides real uncertainty, and it's the wrong choice on rainy days, peak meal times, and for long-distance deliveries where variance is naturally larger. Designing the system to produce a calibrated distribution and then making product decisions about how to surface it is fundamentally better than training a point predictor and hoping the mean is useful.\n\nThree approaches to distributional prediction cover the practical space.\n\n**Quantile regression** trains separate models (or separate heads on a shared model) for each target quantile using the pinball loss. It's the simplest approach, works well with GBDTs, and is easy to calibrate per quantile. The downside is that the quantiles aren't guaranteed to be monotonic (p70 could be below p50 on an individual prediction), and the distribution is only represented at the quantiles you trained on.\n\n**Mixture density networks (MDN)** train a neural network whose output parameterizes a mixture of parametric distributions (typically Laplace or Gaussian). Any quantile can be computed analytically from the mixture parameters, and the distribution is fully specified rather than just sampled at known quantiles. MDNs are more expressive but trickier to train: the NLL loss can be unstable, and mode collapse (where all mixture components learn the same mean) is a real risk without careful initialization and regularization.\n\n**Conformal prediction** wraps any point predictor and produces a calibrated interval by calibrating against a held-out set. The interval is guaranteed to have the right coverage marginally, independent of the underlying model's quality. The downside is that the intervals are typically wider than model-specific intervals, because conformal doesn't exploit the model's confidence signals.\n\n\n| Approach | Strength | Weakness | Best For |\n|---|---|---|---|\n| Quantile regression | Simple, GBDT-compatible, directly calibrated | Quantile crossing, fixed quantile set | Baseline, tabular features |\n| Mixture density | Fully distributional, any quantile | Training instability, mode collapse risk | Advanced, deep stack |\n| Conformal | Coverage guarantee, model-agnostic | Wide intervals, slow calibration | Wrapper on top of point model |\n| Ensembles | Variance from disagreement | Expensive at serving | Uncertainty decomposition |\n\n\nThe production choice for ETA at scale tends to be quantile regression for the baseline and MDN for the advanced model, with conformal used as a post-hoc calibrator to correct any residual miscalibration. Ensembles are valuable offline for understanding where the model is uncertain and where disagreement matters, but they rarely survive to production as live-serving paths because the cost multiplies linearly with ensemble size.\n\n\n```mermaid\nflowchart LR\n IN[Route +
Features]:::primary --> M1[Quantile
GBDT]:::orange\n IN --> M2[MDN
Head]:::orange\n IN --> M3[Point
Model]:::orange\n M1 --> Q[p10, p50,
p90 outputs]:::green\n M2 --> D[Mixture
params]:::green\n D --> Q2[Any quantile
computed]:::green\n M3 --> CONF[Conformal
Calibrator]:::teal\n CONF --> INT[Interval with
coverage guarantee]:::green\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\nOnce the model produces a distribution, the product decision is which point to surface. Showing p50 means half of deliveries run late against the ETA, which feels like broken promises. Showing p90 produces obviously padded ETAs that depress order conversion. The operational sweet spot varies by product; most food-delivery platforms land between p65 and p75 for the customer-facing ETA, with the window adapted dynamically based on observed lateness rates. The same model powers dispatch (which uses p50 for optimal courier selection) and customer messaging (which uses p65+), which is only possible because the model emits a full distribution rather than a single number.\n\n### 6.3 Multi-Leg Prediction and Dwell Time\n\nThe naive framing of \"predict total delivery time\" as a single regression target is worse than the leg-wise decomposition in several ways, and understanding why is the heart of the ETA design problem.\n\nFirst, the legs have very different feature dependencies. Pickup drive depends on the road network, traffic, and courier behavior. Dwell depends on the merchant's current order queue, the order's complexity, and the menu items. Drop-off drive depends on a different road network (different starting point), different traffic (potentially in a different neighborhood), and customer-side features like building access complexity. A single model trying to learn all three either wastes capacity on feature interactions that don't apply, or develops gated components that effectively reimplement leg decomposition internally, but with less interpretability.\n\nSecond, the legs have very different noise characteristics. Drive-time variance is roughly proportional to driving distance: a 2km drive has standard deviation around 45 seconds, a 20km drive has standard deviation around 5 minutes. Dwell-time variance is dominated by the merchant and has a heavy tail unrelated to distance: some restaurants have a standard deviation of 2 minutes, others have 12 minutes because they batch orders inconsistently. Summing two noise sources that live in different regimes requires knowing them individually to do variance propagation right.\n\nThird, iteration speed matters. The merchant ops team wants to improve prep-time prediction by adding features specific to the restaurant side: kitchen size, menu complexity, staffing levels. Those features don't belong in the drive-time model. Keeping the two models separate means the merchant team can retrain the dwell model on its own cadence without regression testing against the drive-time model, which is a major productivity multiplier.\n\nThe dwell-time model itself deserves focused attention because it's the component with the highest headroom in most ETA systems.\n\n\n```mermaid\nstateDiagram-v2\n [*] --> PLACED: Order placed\n\n PLACED --> ACCEPTED: Merchant accepts\n ACCEPTED --> PREPARING: Kitchen starts\n\n PREPARING --> READY: Food ready\n\n READY --> HANDED_OFF: Courier arrives
+ picks up\n\n HANDED_OFF --> [*]\n\n classDef placed fill:#ffa94d,stroke:#000,color:#000\n classDef accepted fill:#00ceff,stroke:#000,color:#000\n classDef preparing fill:#38d9a9,stroke:#000,color:#000\n classDef ready fill:#69db7c,stroke:#000,color:#000\n classDef handed_off fill:#69db7c,stroke:#000,color:#000\n\n class PLACED placed\n class ACCEPTED accepted\n class PREPARING preparing\n class READY ready\n class HANDED_OFF handed_off\n```\n\n\nDwell decomposes into three sub-intervals: acceptance lag (order placed to merchant acceptance), prep time (acceptance to ready), and hand-off lag (ready to courier pickup). Each has distinct drivers. Acceptance lag depends on whether the merchant is using the tablet actively; prep time depends on the kitchen; hand-off lag depends on courier arrival timing relative to food readiness. A single dwell model predicts the whole interval conditional on features from all three sub-sources, but some platforms decompose further and predict each sub-interval separately when the data supports it.\n\nThe merchant-prep sub-model is typically a small GBDT that takes merchant features (rolling prep-time percentiles conditioned on day and hour), order features (item count, menu categories, customizations), and current-queue features (how many pending orders this merchant is already preparing). The last one is the single biggest feature in the sub-model: a merchant with 2 pending orders at 6:30pm is fundamentally different from the same merchant with 12 pending orders. Adding queue depth as a feature typically cuts prep-time MAE by 15 to 25% over a model that ignores it.\n\nOne subtle trap in dwell modeling is that the prep-time label is right-censored by courier arrival. If the courier arrives before the food is ready, the observed \"ready time\" is actually \"hand-off time,\" which over-states prep time. If the food is ready before the courier arrives, the observed ready time is accurate. Training on the observed hand-off timestamp without adjustment biases prep-time predictions upward by 10 to 30 seconds in peak hours. The fix is to have merchants tap a \"ready\" button at the actual ready moment, which gives a clean label but depends on merchant compliance, or to model hand-off lag separately and subtract it, which is more robust but requires both signals.\n\n### 6.4 Mid-Trip Refinement and Adversarial Dynamics\n\nThe initial ETA at order time is a prediction made from limited information: the courier hasn't been dispatched yet, traffic on the specific route hasn't been sampled from this courier's perspective, and the merchant's current queue depth is an estimate. As the trip progresses, real information accumulates, and the live ETA needs to integrate that information gracefully.\n\nThe refinement policy is conceptually simple. Every 10 to 30 seconds, re-run the scorer with the latest features, which now include the courier's current position (updating the remaining pickup or drop-off distance), the latest segment speeds on the upcoming route, and any state transitions that have fired (merchant accepted, food is ready, courier arrived at merchant). The output is a new ETA distribution, and the customer sees a refreshed countdown.\n\nThe subtlety is how to handle large updates. If the new ETA is 3 minutes later than what the customer was shown 10 minutes ago, displaying the correction feels like breaking a promise. If the new ETA is 5 minutes earlier, the customer might not be ready to receive the food. Most platforms damp the displayed ETA with a moving average so it only steps in small increments (at most 60 seconds per minute of wall-clock), while the internal dispatch ETA remains the raw model output. The damped display is a product layer, not a model layer, but it has to be designed together with the model because the damping imposes a constraint on how fast the underlying model can change without violating the display policy.\n\n\n```mermaid\nflowchart LR\n T0[t=0
Order placed]:::primary --> P0[Initial ETA
from features
at order time]:::orange\n P0 --> DISP[Customer sees
ETA = 32 min]:::green\n T1[t+30s
Courier dispatched]:::primary --> P1[Re-score
with courier
+ live route]:::orange\n P1 --> UPD1[Update 1:
30 min]:::green\n T2[t+2min
Traffic refresh]:::primary --> P2[Re-score
with fresh
segment speeds]:::orange\n P2 --> UPD2[Update 2:
31 min]:::green\n T3[t+12min
At merchant]:::primary --> P3[Re-score with
observed prep
+ drop route]:::orange\n P3 --> UPD3[Update 3:
18 min
remaining]:::green\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\nA separate concern is adversarial behavior within the fleet. Couriers are rewarded (explicitly through pay, implicitly through ratings) for fast deliveries, and if the ETA model becomes part of the reward function, couriers have incentives to drive in ways that game the model rather than deliver fast. The classic example is a courier who detours through a fast back road but doesn't log through the app's map, making it look like they drove a longer nominal route in less time. If the model treats this as \"this courier is fast on suburban routes,\" it might preferentially dispatch them on routes where the shortcut doesn't exist, producing worse outcomes.\n\nThe defense is to decouple courier-specific features from the operational model. Courier speed priors are useful but should be computed from GPS-observed speeds on common segments relative to the fleet average, not from observed total trip times. A courier who is reliably 10% faster on suburban residential streets gets a positive prior for residential-heavy routes; a courier whose total trip times are 10% below average without a segment-level explanation doesn't get a prior adjustment because the signal could be coming from unobserved shortcuts or from cherry-picking easy trips.\n\nRelated is the feedback loop between ETA and dispatch. Dispatch picks couriers based on predicted pickup time, which means couriers closer to the restaurant are preferred, which biases the training data toward \"close couriers\" and might under-sample training examples of longer pickups. The fix is random exploration: a small fraction of dispatches (typically 1 to 3%) override the dispatch decision with a randomized pick that includes couriers further from the restaurant. Those dispatches are more expensive in pickup time but generate training data that keeps the model calibrated across the full distance distribution. Without the exploration, the model slowly becomes biased toward short pickups and its error on long ones grows until the dispatcher starts mispicking even under normal operation.\n\n---\n\n# 7. Evaluation and Iteration\n\n### Offline Evaluation\n\nOffline evaluation uses temporal holdout (train on weeks 1 through 4, validate on week 5, test on the most recent complete week) with three headline metrics: MAE, quantile loss at the served quantile, and coverage of the 80% prediction interval. Aggregate numbers hide the interesting regressions, so slice evaluation is mandatory across city, hour-of-day band, distance band, weather condition, and distance from the training geography.\n\nThe specific slices that pay for themselves are meal peaks versus off-peak (peaks have 2-3x higher variance and different feature sensitivities), rain versus clear weather (traffic patterns are discontinuously different), long-tail distance bands (routes beyond 10km are rare but produce the biggest absolute errors), and new-merchant bucket (merchants with under 100 historical orders have unstable prep-time priors and often regress on new model versions).\n\nA candidate model promotes only when it beats the champion on aggregate MAE and on every major slice within a tolerance of 2%. Strict slice gates sometimes block genuinely better models that regress modestly on one slice while improving many others, which is the right trade-off because slice regressions manifest as localized customer-experience issues that take months to rebuild trust after.\n\nCalibration evaluation runs on the distributional outputs. The empirical coverage of the 80% interval should be 80% across the full test set and within 2 percentage points across each major slice. Deviation patterns tell a specific story: systematic under-coverage means the model is overconfident on uncertain cases, systematic over-coverage means the intervals are too wide and the customer-facing ETA is more padded than it needs to be. Both are fixable, but they require different root-cause investigations.\n\n### Online Evaluation\n\nOnline evaluation uses A/B testing at the dispatch-unit level. The assignment unit is the dispatch request (courier-order pair) rather than the user or the order, because the ETA model's immediate effect is on dispatch decisions and downstream effects (lateness, customer complaints) propagate from there. Running an A/B at the order level would conflate dispatch changes with subsequent ETA refreshes.\n\nThe primary A/B metrics are lateness rate (fraction of deliveries exceeding customer ETA by 5+ minutes), over-promise rate (fraction where actual time exceeds ETA at all), dispatch quality (fraction of dispatches where the chosen courier was within 2 minutes of the predicted pickup time), and conversion rate at checkout (catches over-padded ETAs). The guardrail metrics are customer complaint rate, courier utilization, and serving latency.\n\nThe rollout ramp is faster than for fraud because the feedback is fast. A typical rollout starts at 1% of dispatches, moves to 10% after 48 hours if guardrails are clean, then 50% after 72 hours, then 100% after another week. Rollbacks are triggered by lateness rate exceeding the champion's by more than 10 relative percent or by any guardrail metric regressing past its pre-set threshold.\n\nThe trickiest measurement problem in online ETA evaluation is long-term effects. A model that over-pads ETAs today might show no short-term complaint-rate change but slowly erodes conversion over weeks as customers compare against competitor platforms. Guardrails on conversion have to run on multi-week windows, which slows the decision cycle for changes that could affect it. The standard compromise is to ramp to 50% and hold for 4 weeks rather than 1 before final promotion, paying a slower rollout for a more reliable conversion signal.\n\n### Monitoring\n\n\n| Signal | What It Catches | Alarm Threshold |\n|---|---|---|\n| MAE over rolling 1-hour window | Model corruption, feature pipeline outage | >15% above rolling 24-hour baseline |\n| Interval coverage per slice | Miscalibration, drift, distribution shift | Coverage outside [75%, 85%] for a major slice |\n| Feature freshness (traffic) | Segment-speed aggregator lag | P99 segment speed older than 5 minutes |\n| Feature freshness (merchant) | Merchant prep aggregator lag | Merchant feature older than 3 hours |\n| Training-serving skew | Point-in-time join errors, pipeline divergence | Top-20 features by gain, any >1% serving-time divergence |\n| Lateness event rate | Business outcome; fast feedback | Hourly rate above 15% for any major city |\n| ETA bias (mean residual) | Systematic over- or under-prediction | Rolling mean residual outside [-30s, +30s] |\n| Customer complaint rate | False-promise waves, model regressions | >200 complaints per million orders |\n| Subgroup lateness ratio | Fairness drift across neighborhoods | Max/min ratio across zip-code deciles >1.5 |\n| Refresh-correction size | Poor initial predictions, signal gap | Avg mid-trip correction >60 seconds |\n\n\nTraining-serving skew is the monitor that earns its keep most often, just like in fraud. A feature that's mean zero in training but mean 0.05 at serving silently shifts every prediction by a small amount, and since ETA has no single hard threshold (unlike fraud's decline), the error hides in aggregate MAE for days. The defense is a shadow recomputation job that reproduces serving-time features from logged raw inputs and compares them to what the online feature store served. Any top-20 feature diverging by more than 1% pages the on-call.\n\nDrift in the lateness rate itself is the business-level signal that everything else rolls up to. When lateness spikes, the triage question is always \"is this a model regression, a feature-pipeline outage, or an underlying operational shift?\" Model regressions and pipeline outages show up in the training-serving skew monitor and the feature freshness monitors before lateness moves. Genuine operational shifts (a city just added 500 couriers, a weather event is causing widespread delays) show up in the feature distribution monitors and in raw operational metrics (dispatch volume, courier utilization). Distinguishing the two categories quickly is what keeps an incident from becoming an outage.\n\nRetraining cadence is daily for the full stack and hourly for incremental updates to the segment-speed encodings. The daily retrain takes about 4 hours end-to-end on a mid-sized GPU cluster and another 2 hours for shadow evaluation before promotion. Emergency retrains (novel geographic rollouts, sudden infrastructure changes) run on a shortened 90-day window of recent data and a compressed 4-hour shadow.\n\n---\n\n# Interview Questions\n\n**Q1: Why predict the legs of a delivery separately instead of training one model to predict the total delivery time?**\n\nThe legs have different feature dependencies, different noise characteristics, and different iteration cadences, and combining them into one model forces compromises on all three. Pickup and drop-off drive times depend on the road network and traffic but not on merchant features; dwell time depends on the merchant and the order but not on the road network. A single model either wastes capacity learning feature interactions that don't apply or develops internal gating that effectively reimplements leg decomposition with worse interpretability. Drive-time variance scales with distance while dwell-time variance is dominated by the merchant; summing the two correctly for uncertainty propagation requires modeling them individually. The separate-model design also lets the merchant ops team iterate on prep-time prediction without regression-testing against the drive-time stack, which is a major productivity gain.\n\n**Q2: How do you quantify uncertainty in ETA predictions, and why does it matter beyond just showing the customer a range?**\n\nQuantile regression, mixture density heads, and conformal prediction are the three workhorse approaches. Quantile regression trains separate heads at each target quantile using the pinball loss and is the simplest approach on tabular models. Mixture density networks output parameters of a mixture of Laplaces or Gaussians and produce fully distributional predictions, but they're trickier to train. Conformal wraps any point predictor and guarantees marginal coverage. Uncertainty matters beyond the customer display because dispatch uses the median for optimal courier selection, pricing uses the mean for unbiased expected-cost calculations, and customer-facing UX uses a higher quantile (p65 to p75) to balance lateness complaints against over-padding. A single point predictor forces all three consumers to share one number, which necessarily compromises at least one of them. A distributional output lets each consumer pick its own point without retraining.\n\n**Q3: When is a GNN over the road network actually worth it versus a GBDT with hand-crafted route features?**\n\nA well-tuned GBDT with engineered features (total distance, signal count, road-class distribution, route-weighted traffic speed) captures most of the signal on common routes and beats a GNN on cost by an order of magnitude. The GNN earns its place in three specific situations: routes through under-represented geographies where training data is sparse, novel segments that haven't been seen in training, and long routes where two-hop graph context (what's around the segments the courier is about to enter) matters more than single-segment features. In production at a mature platform, the two coexist: the GBDT handles high-volume common routes with sub-5ms serving, and the GNN either feeds embeddings as additional features into the GBDT or scores a fraction of traffic where the segment-based features are known to be weak. The GNN rarely replaces the GBDT outright because the cost-accuracy trade-off doesn't justify it on the bulk of traffic.\n\n**Q4: How do you prevent a feedback loop between the ETA model and the dispatch system from degrading training data?**\n\nDispatch picks couriers based on predicted pickup time, which biases the training data toward short-pickup trips and under-samples examples where the courier was further from the restaurant. Over time the model's error on long pickups grows because it's rarely seeing them, and eventually the dispatcher starts mispicking even under normal operation. The standard fix is random exploration: 1 to 3% of dispatches override the ranked choice with a randomized pick that includes couriers further from the restaurant. Those trips are more expensive in pickup time but generate training examples across the full distance distribution, keeping the model calibrated. A related concern is courier-specific features: a courier whose total trip times look fast without a segment-level speed explanation might be cherry-picking easy trips or using off-map shortcuts, so courier priors are computed from GPS-observed segment speeds relative to the fleet average rather than from total trip durations.\n\n**Q5: Why is temporal censoring a problem in ETA training data, and how do you handle it?**\n\nDeliveries that run very late sometimes get cancelled, reassigned, or resolved manually, and those trips don't appear in the completed-delivery training set. The observed distribution is censored on the right tail: the worst outcomes are systematically missing. Training on the censored distribution biases the model toward optimism because the examples that would pull the upper quantiles to the right have been removed. A second form of censoring affects dwell time specifically: prep-time labels are bounded above by courier arrival time, so trips where the courier arrives early produce artificially inflated \"ready\" timestamps. The fixes are censoring-aware losses (Tobit regression or a modified quantile loss that treats censored examples as lower bounds on the true duration), explicit \"ready\" signals from the merchant to de-censor prep time, and retaining censored trips with a censoring indicator so the model can learn to handle them rather than just dropping them. Ignoring censoring is one of the most common reasons an ETA model's p90 quantile is systematically under-coverage.\n\n---\n\n# Summary\n\n- ETA prediction is a per-leg regression problem with distributional outputs, not a single-model end-to-end time predictor, because the legs have different feature dependencies, noise characteristics, and iteration cadences.\n- The natural decomposition for on-demand delivery is pickup drive, merchant dwell, drop-off drive; each leg gets its own model sharing infrastructure but with separate features and separate retraining cycles.\n- Quantile loss and mixture density networks both produce calibrated distributional outputs, which lets dispatch use the median, pricing use the mean, and customer UX use p65-p75 from the same model without retraining.\n- Road-network representations split between segment-based features (cheap, interpretable, fast) and graph-based GNN encodings (better on rare routes and novel segments); the honest answer is a hybrid, with the GBDT as the fast path and the GNN supplementing on known-weak slices.\n- Serving has a 150ms p99 budget at dispatch scale with 50K req/s peak; feature fetch dominates the budget, and the three leg models run in parallel after features land.\n- Labels arrive in seconds (drop-off confirmation), which supports daily retraining and hourly incremental updates to traffic-derived features; the main label quality issue is right-censoring on the tail, which biases the model toward optimism if not explicitly handled.\n- Mid-trip refinement re-scores every 10 to 30 seconds based on GPS pings and state transitions; the displayed ETA is damped by a product layer so customers see smooth updates while dispatch sees the raw model output.\n- Evaluation combines offline temporal-holdout MAE and calibration coverage with online A/B on lateness rate, over-promise rate, and conversion; slice dashboards across city, meal peak, weather, and distance band catch regressions that aggregate metrics hide.\n\nNext we turn to notification relevance, which shares ETA's concern for user trust but flips the design problem: instead of predicting a quantity, the system decides whether to send a message at all and when, trading user engagement against the risk of notification fatigue.","pageType":"ml-system-design"}

Design an ETA Prediction System

Ashish Pratap Singh

Get Premium