Uber

What the interviewer is probing

This question tests structured thinking about data sparsity as a modeling problem: the candidate must reason about what signals remain useful when the primary signal is absent, how to design an input representation that degrades gracefully rather than catastrophically, and whether the fallback behavior should be engineered or learned. It also probes evaluation maturity around the dense-sparse performance trade-off.

What a strong answer covers

Must establish

• When direct segment observations are sparse or absent, useful signal still exists at multiple spatial scales, such as nearby road segments, broader geographic regions, and historical patterns for this segment at this time of week, and the model must be able to draw on those fallback signals without special-casing.
• The input representation must preserve the distinction between a segment observed to be fast and a segment not observed at all, and it must degrade gracefully across coverage regimes, so the model treats sparsity as information to weigh rather than a special case to route around.
• The blending of local versus regional signal must be learned from data rather than hard-coded, and the candidate must articulate what learns it and how the model behaves when local observations vanish.

A strong answer adds

• The candidate explicitly designs the fallback hierarchy: segment-level recent observations, then graph-neighborhood aggregates, then broader regional aggregates, then day-of-week historical baselines, and explains what each level provides when the level above it is empty.
• The candidate recognizes that evaluation must be stratified by observation density: a single aggregate metric will be dominated by dense segments and will not reveal whether sparse-segment performance improved.
• Cold-start for a brand-new market is addressed explicitly: the model relies entirely on regional and historical priors initially, and the candidate describes what triggers the transition toward local observations and how quickly that happens.
• The candidate notes that sparse segments are not randomly distributed: they tend to cluster in certain geographies and time windows, so a model that learns to handle them well in one sparse market will generalize to new sparse markets without retraining.

Exceptional depth

• The candidate notes that graph-neighborhood aggregates for sparse segments may themselves be computed from sparse observations, creating a propagation problem, and proposes a confidence-weighted aggregation that discounts neighborhoods with low coverage rather than treating all neighbor signals equally.

Common misses

• Proposes imputing missing observations with zeros or means before the model sees them, which destroys the distinction between a segment that was observed to be fast and one that was not observed at all.
• Hard-codes the fallback logic, for example always using regional averages when segment coverage is below a threshold, rather than letting the model learn which signals are informative in context.
• Designs separate models for dense and sparse segments, which breaks generalization to new markets and requires an explicit routing layer that itself must be maintained.
• Evaluates the redesign on aggregate metrics dominated by dense segments and declares success without examining sparse-segment performance separately.
• Does not address the cold-start case for new markets, implicitly assuming historical data always exists.

The underlying concept

Sparsity in spatial forecasting is a missing-data problem with structure: when a segment lacks direct observations, correlated signals at nearby segments, broader regions, and historical patterns for the same time of week carry real information. One robust design provides the model with a complete, fixed-size input that represents all these signal levels, rather than feeding raw observations whose count varies; other viable designs reach the same property through explicit missingness indicators or graph message passing. This way the model sees the same input shape regardless of coverage and learns, through training on a mix of dense and sparse examples, to weight the available signals appropriately. The alternative, hard-coding fallback rules, tends to fail on novel geographies where the right fallback level is not obvious in advance.

Source

Derived from Scaling Real-Time Traffic Forecasting with a Graph-Aware Transformer

What the interviewer is probing

This question tests whether the candidate can reason about multi-stage ML pipelines where metrics at one layer do not compose cleanly into metrics at the next. It probes calibration-versus-resolution decomposition, the specific failure mode of training a downstream model on a distribution that then drifts, and the architecture of a real-time correction layer. Strong candidates surface the compounding effect of segment-level miscalibration across many segments summed into a trip estimate.

What a strong answer covers

Must establish

• Segment MSE and trip MAE measure different things: MSE rewards resolution, MAE rewards calibration, and a model can improve one while worsening the other because summing miscalibrated segment forecasts across many segments amplifies systematic bias into large trip-level errors.
• When the downstream arrival-time model is retrained on the new forecasts, it learns to correct for the new model's calibration signature. If that signature then drifts post-deployment, the downstream model is correcting for the wrong bias, making things worse rather than better.
• The fix requires continuous calibration of segment-level forecasts before they reach downstream consumers, not a one-time correction at training time, so that the distributional contract the downstream model was trained on stays stable over time.

A strong answer adds

• Decomposing error into resolution and calibration components early in model development allows the team to distinguish 'this model has more information' from 'this model is consistently wrong in a correctable direction', and to project segment-level gains into trip-level impact before any A/B test.
• A real-time calibration layer should bucket forecasts by city and time window, join against recent observed traversal times, and apply a learned correction per bucket. The correction rate and bucket granularity are tuned to balance responsiveness against noise sensitivity.
• The calibration layer's own failure modes need monitoring: if ground truth observations are delayed or sparse for a city, the correction can lag or overcorrect. A fallback to a recent stable correction or a wider bucket is needed.
• The downstream model should be retrained on calibrated forecasts, and its evaluation should include a holdout period after a simulated retraining event to detect the specific pattern of post-retraining drift degradation.

Exceptional depth

• Treating calibration drift as a covariate shift problem and applying techniques such as importance weighting or continual learning on the segment model itself, rather than post-hoc correction, addresses the root cause rather than the symptom, but requires significantly more infrastructure.
• Formalizing the downstream model's dependency as a contract (the forecast distribution it was trained on must remain stationary) and monitoring that contract with a distribution test on forecast residuals provides an automated early-warning system before rider-facing metrics degrade.

Common misses

• Attributes the trip-level degradation to the downstream model being stale rather than to calibration drift in the upstream forecasts, missing that the downstream model's staleness is a consequence of the upstream shift, not an independent cause.
• Conflates resolution with calibration and proposes improving the segment model further instead of diagnosing the systematic bias, treating a calibration problem as a variance problem.
• Designs the calibration layer at trip granularity rather than segment granularity, failing to correct the errors before they are summed and making the correction harder to generalize.
• Does not account for heterogeneity across cities and times of day, proposing a single global correction factor that masks large local miscalibrations.
• Skips monitoring the calibration layer itself and assumes it self-corrects, leaving a latent failure mode when ground truth observations are sparse or delayed in thin markets.

The underlying concept

In a multi-stage prediction pipeline, each stage's output becomes the next stage's training distribution. If the upstream model's output distribution drifts after the downstream model is trained, the downstream model applies corrections learned for a distribution that no longer exists, degrading performance even when the upstream model's raw accuracy improves. This is a form of dataset shift induced by the system's own update cadence. The resolution-calibration decomposition is a useful diagnostic: resolution measures how much real variance a forecast explains and is hard to fake, while calibration measures systematic bias and is easy to correct post hoc. Correcting calibration continuously, rather than once at training time, keeps the upstream contract stable and prevents compounding errors across a pipeline. The architecture lesson is that any layer whose outputs feed another learned model must treat its output distribution as a first-class contract to be monitored and maintained.

Source

Derived from Scaling Real-Time Traffic Forecasting with a Graph-Aware Transformer

What the interviewer is probing

This question tests whether the candidate can bridge ML prediction with combinatorial optimization at scale — recognizing that 'pick the highest-uplift treatment per user' is wrong when budgets couple decisions across millions of users. It probes metric judgment (cross-vertical ROI, strategic weights), architecture (prediction layer decoupled from valuation layer, budget feedback control), and scale-and-failure reasoning (solver choice, time-series cost complexity, pacing under model drift). Strong candidates will identify the NP-hard nature of the assignment problem and reason about the solver trade-off without needing to know CP-SAT by name.

What a strong answer covers

Problem framing - Clarify: How many users, levers, budget buckets? What's the assignment cadence? Are assignments one-per-user or can a user receive multiple incentives? - The core constraint: this is a multiple knapsack problem — knapsacks are team budgets, items are user-incentive pairs with predicted cost and value, goal is to maximize total value subject to per-knapsack budget constraints. - Naive per-user greedy (assign each user their highest-uplift option) ignores coupling: high-cost incentives burn budget that could serve more users at lower cost.

ROI signal design - Wrong turn: predict a single engagement metric per incentive. Fails because a delivery incentive that pulls a driver away from rides improves one metric while hurting another — net effect is negative. - Better: predict a vector of metric uplifts (rides trips, eats orders, driver utilization, earnings per hour) using uplift models trained on randomized experiment data. - Apply a strategic weight vector — set by business context — to convert the metric vector to a scalar ROI via dot product. Weights for a new market expansion emphasize growth; mature markets emphasize efficiency. Separating prediction (objective) from valuation (subjective) lets the same model serve different strategic priorities without retraining. - Guardrails: include cross-vertical cannibalization as a negative-weight term. Monitor net marketplace metrics, not just primary metrics, in holdout evaluation.

Budget pacing across time - Problem: quarterly budgets with weekly or daily assignment cycles. ML cost predictions drift; if predictions underestimate cost, you overspend early. - Control loop design: at each cycle, compute remaining_budget = configured_budget - observed_spend - predicted_liability, where predicted liability is the forecasted future cost of already-assigned active incentives not yet paid out. - This self-corrects: if actual costs exceed predictions, the pacer tightens the cap for future cycles, smoothing over the quarter. - Trade-off on cost modeling: modeling cost as a daily time-series over the incentive duration is accurate but explodes solver dimensionality (N users × T days constraints vs. N scalar constraints) and creates artificial bottlenecks when a predicted spike on one day blocks otherwise-viable assignments. Collapsing total predicted cost to the assignment day sacrifices granularity but makes the solver tractable and dramatically improves budget utilization.

Optimizer design - Scale challenge: with millions of users and hundreds of levers, the candidate space is the Cartesian product — potentially billions of user-incentive pairs. - Why LP fails at scale: LP relaxes the binary assignment decision to a continuous variable, then rounds. For large combinatorial problems the LP relaxation's solution is far from integer-feasible, and branch-and-bound search explodes. Runtime can exceed 24 hours on 100K users. - Better fit: constraint programming solvers (e.g., CP-SAT) natively handle discrete, binary assignment variables and use propagation + intelligent pruning to eliminate infeasible branches early. Same 100K-user instance solved in minutes. - Plug-and-play solver interface: abstract the problem definition (variables, constraints, objective) from the solver implementation so you can swap solvers as problem structure evolves without rewriting business logic. - Common wrong turn: trying to scale a greedy heuristic (rank by ROI/cost ratio, assign greedily until budget exhausted). This is fast but suboptimal and provides no optimality guarantees or constraint flexibility.

System architecture - Offline/batch flow: cron-triggered orchestrator → fetch eligible user pools from segmentation → generate user × lever candidate pairs → score with ML platform (uplift, cost) → pacer computes available budget cap → optimizer solves MKP → push assignments to downstream execution services. - Latency: optimization is offline/batch, so SLA is minutes to hours, not milliseconds. Decouple from real-time serving. - Scale pressure points: candidate generation (Cartesian product can be billions of rows — pre-filter ineligible pairs before scoring), ML scoring (batch inference at scale), solver memory (large binary programs; may need to shard by geography or time window). - Failure modes: solver timeout (set a time limit, return best feasible solution found); ML model staleness (monitor feature drift; degrade gracefully to simpler heuristic if model is stale); budget overrun from late-arriving cost signals (pacer's predicted liability must account for settlement lag).

Evaluation - Offline: simulate allocations on historical data; compare total assigned ROI and budget utilization vs. FIFO baseline. - Online: holdout experiment — randomly assign a fraction of users to the optimizer vs. baseline; measure net marketplace metrics (trips, earnings, utilization) per dollar spent, with guardrails on cross-vertical cannibalization. - Exploration vs. exploitation: current system requires manual intervention to test new incentive structures. Flag as open problem: auto-allocate a small exploration budget to novel incentives, gather uplift signal, retrain models, inject validated structures — closing the loop.

The underlying concept

Incentive allocation at scale is a multiple knapsack problem (MKP): you must assign binary decisions (give user U incentive I or not) to maximize a global objective subject to coupling budget constraints, which makes the decisions across users non-independent. This is fundamentally combinatorial and NP-hard, which is why greedy or LP approaches break down as the problem scales — LP's continuous relaxation poorly approximates the integer solution when the problem is highly combinatorial, while constraint programming solvers exploit problem structure (propagation, pruning) to find near-optimal integer solutions efficiently. The two-layer ROI design — predicting metric uplifts separately from applying strategic weights — is a classic separation of concerns: the prediction layer is a calibration problem (train once, reuse), while the valuation layer is a business policy problem (change without retraining). Budget pacing as a feedback control loop is the standard engineering answer to the tension between probabilistic ML predictions and hard financial constraints: rather than trusting the model's cost forecast, continuously reconcile predictions against observed reality and adjust the constraint fed into future optimization runs.

Source

Derived from Beyond Prediction: Solving the Multiple Knapsack Problem at Scale: How Uber Optimizes Incentives

What the interviewer is probing

This question probes whether the candidate can evaluate a spatial partitioning scheme against concrete analytical requirements rather than intuitive appeal — specifically, the trade-offs between uniform and irregular cells for statistical aggregation, gradient estimation, and cross-city comparability. It also tests whether the candidate can identify marketplace interference as a confound when designing an experiment over a shared supply pool, and whether they can separate operational aesthetics from modeling requirements.

What a strong answer covers

• Frame the decision: the grid system is infrastructure for aggregation — bucketing events to estimate local supply/demand ratios. The right question is: which scheme produces more accurate, comparable, and stable surplus/deficit signals, not which looks more intuitive on a map.
• Properties of uniform hexagonal cells worth naming:
• Equal area across cells means raw counts are directly comparable; no need to normalize by cell size.
• Single neighbor-distance type (all six neighbors equidistant) means spatial smoothing and gradient estimation use a single coefficient set — the same kernel applies everywhere.
• Cell assignments are deterministic from lat/lng and resolution; no lookup table, no stale boundary files.
• Hierarchical rollup is a bitwise operation; coarsening is O(1) per point.
• Problems with neighborhood/zip-aligned zones:
• Irregular area: a downtown zip covers a few blocks; a suburban zip covers tens of square miles. Raw event counts are incomparable; you must normalize by area or expected demand density, introducing its own estimation error.
• Two neighbor-distance types (shared-edge vs. shared-vertex neighbors) complicate spatial smoothing — the same algorithm behaves differently depending on adjacency type.
• Boundaries are externally controlled and change (redistricting, ops redrawing zones); this creates discontinuities in time series.
• Cross-city comparability breaks down — surge thresholds calibrated in one city's zone structure don't transfer to another.
• When adaptive/irregular cells might legitimately win: if the primary use case is human interpretation by ops teams (not modeling), irregular zones have real value. Separate that concern — consider overlaying ops-friendly labels on top of the hexagonal grid rather than replacing the grid.
• Gradient and motion effects: a rider or driver moving through a city generates events near cell boundaries. Hexagons minimize boundary length relative to area, reducing the frequency of edge-case assignments. Irregular zones with jagged or elongated shapes can place high-traffic corridors near boundaries, inflating noise in cells that happen to straddle a major road.
• Common wrong turn: accepting the PM's framing at face value and proposing a straightforward A/B test without recognizing that surge pricing is a marketplace intervention — treating and control cells compete for the same drivers, so a naive geographic split produces interference bias. A valid experiment requires a cluster-level design (city-level holdout or matched-market pairs) with pre-registered metrics and sufficient power given slow equilibration.
• Recommendation structure: surface three explicit trade-offs — (1) statistical comparability vs. human legibility, (2) boundary stability vs. operational alignment, (3) modeling simplicity vs. granularity flexibility. Recommend keeping the hexagonal grid as the analytical layer and building an optional mapping from hexagons to named neighborhoods for dashboards. If the adaptive scheme is piloted, require a pre-registration of surge accuracy metrics (e.g., driver arrival time, unfulfilled request rate) and a city-pair matched holdout.

The underlying concept

Spatial discretization imposes a fundamental trade-off: cells that are regular and uniform make statistical aggregation straightforward — counts are directly comparable, spatial kernels use a single distance definition, and hierarchical rollup is cheap — while cells aligned to human-meaningful boundaries make outputs interpretable but introduce heterogeneous areas, irregular adjacency structures, and external dependencies on boundary files. For marketplace applications where signals from different zones feed the same pricing or dispatch model, comparability across cells is usually more valuable than legibility, because normalization errors and boundary instability compound over time and across cities. The hexagonal grid's specific advantage is that among tileable polygons, hexagons have the highest area-to-perimeter ratio and a single neighbor-distance type, both of which minimize quantization noise when modeling continuously moving agents. When human interpretability is genuinely needed, the standard engineering answer is to maintain two layers — a regular analytical grid and a named-zone mapping on top — rather than forcing the analytical layer to conform to human geography.

Source

Derived from H3: Uber’s Hexagonal Hierarchical Spatial Index

What the interviewer is probing

This question probes whether the candidate can decompose prediction uncertainty into meaningful components (model/epistemic, distributional shift, irreducible noise) and reason about each one's implications for anomaly detection — not just reach for a confidence interval formula. It also tests calibration thinking (coverage vs. width trade-off), the practical costs of miscalibration in an ops context, and scale/latency reasoning about making stochastic inference feasible at millions of metrics.

What a strong answer covers

• Decompose uncertainty before designing anything. Three distinct sources: (1) epistemic/model uncertainty — reducible with more data, reflects parameter uncertainty; (2) distributional shift / misspecification — the test series looks different from training, irrelevant to how much data you have; (3) irreducible / aleatoric noise — inherent variability in the process. Conflating these leads to systematically wrong intervals: a model confident in its parameters still produces garbage if the input is out-of-distribution.
• Epistemic uncertainty via MC Dropout. Keep dropout active at inference time; run multiple stochastic forward passes; use the sample variance as a proxy for parameter uncertainty. Key design choices: dropout rate is a hyperparameter (tune on a held-out set, not by feel); number of passes trades off variance of the variance estimate against latency (standard error shrinks as 1/√B).
• Misspecification uncertainty via encoder distance. Pre-train an encoder (e.g., LSTM autoencoder) to embed historical windows into a latent space. At test time, apply MC dropout through the encoder too — not just the prediction head. A test window that encodes poorly (high variance across passes in the embedding) signals distributional shift and widens the interval automatically. This is the critical insight: dropout through the encoder, not just the predictor, captures OOD inputs.
• Aleatoric / inherent noise. Estimate residual variance from a held-out validation set (RSS-based estimate). This is a fixed additive term — it floors the interval width even when the model is highly confident. Without it, intervals are systematically too narrow on normal series.
• Combining the three terms. Total predictive variance = model+misspecification variance (from MC passes) + inherent noise (from held-out residuals). Construct the prediction interval assuming Gaussian tails as a starting point; validate the Gaussian assumption and use empirical quantiles if the residual distribution is heavy-tailed.
• Calibration validation. For each interval level α, compute empirical coverage on a held-out test set across many series and time segments. A perfectly calibrated 95% interval contains the true value 95% of the time. Check separately for holiday vs. non-holiday segments — miscalibration during high-variance events is the failure mode that matters most. Over-wide intervals lower precision (alert fatigue); under-wide intervals lower recall (missed outages).
• Alert rule design given the recall/precision trade-off. Don't just threshold at ±1.96σ. Use asymmetric thresholds if downside anomalies (metric drops) are more severe than upside ones. Consider a severity tier: high-uncertainty series (e.g., New Year's Eve) get wider bands automatically, not a manually tuned exception. Log near-misses for post-hoc review.
• Scale and latency. Precompute and cache the encoder embeddings — they change slowly (update every few minutes, not per-request). Run B stochastic passes in batch on GPU or via vectorized CPU operations (random masking is cheap); a few hundred passes at ~10ms per metric is feasible if matrix ops are efficient. Serve inherent noise estimates as static lookup values per metric segment (holiday vs. normal). Separate the training pipeline (daily/weekly retrain) from the inference serving path.
• Common wrong turns. Using only the prediction head's dropout and ignoring the encoder (coverage collapses for OOD inputs). Using training-set residuals to estimate aleatoric noise (optimistic bias). Treating calibration as a one-time check rather than a continuously monitored metric. Setting a single global threshold instead of per-series or per-segment intervals.

The underlying concept

Prediction intervals in neural networks require decomposing uncertainty by cause, not just computing a variance. Epistemic uncertainty (parameter ignorance) can be approximated cheaply via Monte Carlo Dropout — keeping dropout active at inference and treating the ensemble of stochastic outputs as samples from a posterior. Distributional shift (misspecification) is the component most often ignored: it captures the case where the model's training distribution doesn't match the test input, and applying MC Dropout through feature-extraction layers (not just the prediction head) turns the encoder's own uncertainty into a signal. Aleatoric noise, the irreducible process variance, must be estimated from held-out residuals and added as a floor. Calibration — the empirical match between stated coverage and actual coverage — is the auditable output that connects statistical validity to operational decisions like alert thresholds, and it must be evaluated separately on high-variance segments because that is precisely where a poorly calibrated system will fail.

Source

Derived from Engineering Uncertainty Estimation in Neural Networks for Time Series Prediction at Uber

What the interviewer is probing

This question surfaces whether the candidate understands the multiple comparisons problem at a mechanistic level (not just as trivia), can articulate how within-user session correlation inflates false positives beyond the peeking problem alone, and can make a principled recommendation about sequential testing — including its power trade-offs. Strong candidates connect the statistical design to the business decision: what is the cost of a false positive versus a missed regression at 5% rollout versus 50%?

What a strong answer covers

• Why the naive approach fails: A t-test has valid Type I error control only if you commit to a fixed sample size before looking. Each additional peek is an implicit test; across many peeks the probability of ever crossing the threshold by chance — even under the null — grows well above the nominal α. At 95% confidence with frequent peeking, false positive rates can approach 50%, making the test nearly useless as a safety gate.
• The within-user correlation problem: Session-level metrics violate the i.i.d. assumption. A user who is engaged (or disengaged) generates correlated sessions; treating each session as independent underestimates variance, making test statistics appear more significant than they are. This is a separate problem from peeking — it compounds it.
• Correct approach — sequential testing: Use a test designed for continuous monitoring, such as a Sequential Probability Ratio Test (SPRT) or a related method with explicit error-spending functions (e.g., alpha-spending boundaries like O'Brien-Fleming). These methods pre-allocate Type I error across all planned looks, maintaining the global false positive rate.
• Handling within-user correlation: Replace the standard variance estimator with one that accounts for clustering. Delete-a-group jackknife (or bootstrap at the user level) estimates variance by repeatedly dropping user groups and measuring the effect on the estimate — producing a variance estimate robust to within-user correlation without requiring a parametric model of the correlation structure.
• Setting boundaries: The stopping boundary determines the trade-off between early stopping power and Type I error. Conservative boundaries (e.g., O'Brien-Fleming) spend little α early and more late, meaning you need strong evidence to stop early — appropriate when false positives are expensive. More liberal boundaries stop sooner but risk more false alarms.
• Power considerations at low rollout: Sequential tests can have low power at very small sample sizes; early stops based on extreme observed effects may be noise. The business recommendation is: use the test as an alert trigger, not an automatic kill switch, especially below some minimum exposure threshold. A borderline signal at 5% rollout warrants investigation (check for instrumentation issues, segment imbalances) before a rollout decision.
• Common wrong turns: (1) Bonferroni-correcting individual daily tests — this is overly conservative and ignores the sequential structure. (2) Waiting for a fixed horizon 'just to be sure' — defeats the purpose of early stopping. (3) Treating the sequential test boundary as equivalent to a standard p-value threshold — the stopping criteria are not interchangeable.

The underlying concept

Fixed-horizon statistical tests (t-tests, z-tests) have valid Type I error guarantees only when the sample size is determined before data collection and the test is run exactly once. Running the same test repeatedly as data accumulates — 'peeking' — is equivalent to performing multiple correlated hypothesis tests; the probability that at least one exceeds the threshold by chance grows with the number of looks, inflating the false positive rate far above the nominal level. Sequential testing methods solve this by pre-specifying a spending function that allocates the total allowable Type I error across all planned looks, ensuring the global error rate is controlled regardless of when stopping occurs. A separate but related problem arises when observations are not independent — as with session-level metrics from the same users — because standard variance estimators assume i.i.d. data; underestimating variance makes the test statistic appear larger than it truly is. Resampling estimators like the jackknife or bootstrap at the unit-of-randomization level (users, not sessions) produce variance estimates that are robust to this clustering without requiring knowledge of the correlation structure.

Source

Derived from Building an Intelligent Experimentation Platform with Uber Engineering

What the interviewer is probing

Whether the candidate spots that displayed ETAs are simultaneously a prediction and a promise that shapes demand: honest (longer-tail) estimates can be more accurate yet less attractive, suppressing conversion among time-sensitive customers — a selection effect that also changes who orders and thus measured satisfaction. Metric-tree discipline plus mechanism thinking, in Uber's marketplace flavor.

What a strong answer covers

Must establish

• Core hypothesis: accuracy improved partly by raising displayed estimates that were previously optimistic, so customers see honest 40-minute quotes instead of hopeful 30s, and conversion drops among the time-sensitive segment.
• Metric tree discipline: trace browse → restaurant view → cart → checkout (conversion); then delivered experience (quoted vs. actual, lateness) → satisfaction proxies (ratings, support contacts) → reorder; localize which edge declined and when relative to the ETA rollout.
• Selection effect: the remaining orders skew less time-sensitive after time-sensitive customers exit, changing repeat-rate composition so naive before/after comparisons of downstream metrics are not comparable without adjustment.

A strong answer adds

• Distinguish the display effect mechanism: check conversion as a function of displayed ETA, holding merchant and distance constant, pre/post rollout.
• Distinguish the actual-experience effect mechanism: determine whether realized lateness or delivery times worsened for any segment despite average gains.
• Distinguish the measurement/selection mechanism: reorder rates computed on a shifted ordering population require adjustment before they can be compared.
• Segment cuts to localize the story: time-of-day (lunch skews time-sensitive), market density, distance bands, new vs. tenured customers, and merchant type.
• The display-effect story predicts the conversion decline concentrates in segments where displayed ETAs rose most, so segment cuts serve as a directional test.
• The settling experiment: randomize the display policy (e.g., which quantile is shown) while holding the underlying model constant, then measure conversion, realized lateness against promise, satisfaction, and reorder by cohort.
• Decision frame: the answer is rarely to revert to optimistic quotes; it is to choose the displayed quantile that optimizes long-run value by weighing conversions gained by attractive promises against trust lost by broken ones.

Common misses

• Treats accuracy improvement as unambiguously good without recognizing that a displayed ETA is simultaneously a prediction and a promise that shapes demand.
• Does not identify the selection effect: assumes the post-rollout ordering population is comparable to the pre-rollout population when measuring repeat rates.
• Conflates model quality (calibration, error) with display policy (which quantile is shown, how it is framed) and does not propose experimenting on them independently.
• Skips localizing the metric-tree edge where the decline entered, jumping to conclusions without tracing browse through to reorder.

The underlying concept

A displayed prediction is an intervention: customers act on it, so its optimal presentation is a decision-theory problem distinct from its accuracy. Systems that "improved" a model can regress the business by changing the promise distribution — and the composition of who transacts shifts with it, corrupting naive before/after comparisons of downstream metrics (selection on unobserved time-sensitivity). The durable practice: separate model quality (calibration, error) from display policy (which quantile, how framed), and experiment on them independently.

Source

Distilled Prep canon — curated from Uber's public work on ETA prediction and marketplace conversion.

Source: Uber engineering blog

What the interviewer is probing

Hot-partition reasoning: geographic sharding concentrates a local spike onto a handful of shards/keys, so the answer is absorbing structures — caching with request coalescing, aggregation before fan-in, backpressure, and tiered shedding — plus consistency judgment (prices shown can be stale-bounded; the price committed to a trip must be exact) and idempotency under retry storms.

What a strong answer covers

Must establish

• Identify the hot-partition problem: the venue's geo-cells become hot keys in every layer (location ingestion, demand aggregation, pricing reads, matching), and city-scale capacity doesn't help because per-key throughput is the binding constraint.
• Absorption structures on the read path: surge multipliers and ETAs served from replicated caches with short TTLs and request coalescing, so one recompute per cell per tick is shared by thousands of readers rather than triggering thousands of recomputes.
• Consistency split: displayed price and ETA may be bounded-stale (seconds is fine), but the committed trip price must be locked at request time by snapshotting the signed, versioned quote in the trip creation transaction.

A strong answer adds

• Driver and rider events flow through a partitioned log keyed by geo-cell; demand and supply counts are pre-aggregated at the edge of the pipeline so windowed counts, not raw events, cross the network.
• Pricing recomputes per cell on a short cadence and is by design a rate-limited consumer, making it immune to input spikes regardless of how many raw events arrive.
• App refresh jitter and client-side backoff flatten the thundering herd at the source before requests reach backend services.
• Static fallbacks (last-known multiplier with a staleness cap) serve riders if the pricing service lags, preventing a total read failure during the spike.
• Trip creation is idempotent via a client-generated request key and an atomic check-and-create, so rage-taps and retries collapse to one trip.
• Driver offers are similarly idempotent, protecting the matching layer from duplicate dispatch under retry storms.
• Shedding order is explicit: drop analytics and telemetry first, then widen ETA refresh intervals, then coarsen pricing granularity from cell to zone, then queue match requests with honest wait messaging.
• Committed trips and pricing correctness are never shed; the shedding ladder is defined in peacetime so degradation is designed, not discovered.
• Capacity signals (queue depth, cache hit-rate, per-cell RPS) drive automation and dashboards rather than requiring manual intervention.

Common misses

• Treats the problem as a citywide capacity question rather than a per-key throughput question, missing that geographic sharding concentrates the spike onto a handful of hot cells.
• Omits request coalescing, leaving thousands of readers each triggering independent recomputes instead of sharing one compute per cell per tick.
• Does not split consistency requirements, applying the same staleness tolerance to displayed prices and committed trip prices rather than recognizing the committed price must be exact.
• Skips idempotency on trip creation, leaving the system vulnerable to duplicate trips from rage-taps and retries.
• Names no shedding order, or sheds indiscriminately in ways that could drop committed trips or corrupt pricing.

The underlying concept

Geographic load is power-law bursty, and sharding by geography converts a citywide non-event into a per-key crisis — so spike architecture is about absorption: aggregate early (turn N events into one count), coalesce reads (turn N queries into one compute), backpressure producers, and shed along an explicit value ladder. The consistency principle is asymmetric like every marketplace's: advertisements may be stale within bounds, commitments must be exact and idempotent — and the commit point is where the system snapshots truth. Graceful degradation is designed, not discovered: decide in peacetime what fails first.

Source

Distilled Prep canon — curated from Uber's public work on surge infrastructure and real-time marketplace reliability.

Source: Uber engineering blog

What the interviewer is probing

Whether the candidate sees dispatch as a global optimization under latency constraints: greedy nearest-match is locally fine and globally wasteful (assigning a driver to rider A can strand rider B), which is why systems batch into short windows and solve assignment problems. Plus the systems substrate — geospatial indexing of moving objects, staleness tolerance, and simulation-first evaluation for changes too risky to A/B naively.

What a strong answer covers

Must establish

• The objective is multi-term: rider wait (ETA to pickup), completion probability (acceptance and cancel behavior), driver utilization and earnings fairness, and system efficiency; nearest-driver is insufficient because it ignores future demand, driver heading, acceptance likelihood, and the opportunity cost of assignments.
• Matching is batched over short windows (seconds) per area, then solved as a bipartite assignment problem (Hungarian-style or min-cost flow with predicted-ETA edge weights and behavior-adjusted scores); batching converts myopic greed into near-global optima at negligible added wait.
• Candidate generation uses a geospatial index of hexagonal grid cells maintaining driver presence per cell in sharded in-memory state; candidates come from expanding cell rings and are scored with real-time features (road-network ETA, driver state, acceptance model) within tight latency budgets.

A strong answer adds

• GPS noise and location staleness are absorbed by map-matching GPS to the road network and treating locations as estimates with age, scoring with uncertainty and re-checking on offer.
• Retries and declines re-enter the next matching window, and an idempotent offer protocol ensures network flakiness never double-books a driver.
• Reliability is maintained by sharding the matching service by geography (city or region cells) for blast-radius containment.
• A degraded mode falls back to simpler greedy matching locally rather than failing requests outright.
• Surge events are handled by backpressure on window size rather than by bypassing the batching architecture.
• Evaluation follows a ladder: offline simulation replaying historical supply and demand under the new policy is the workhorse, because marketplace interference makes naive A/B misleading; this is followed by switchback experiments in live cities, then a guardrailed rollout with automatic rollback on wait or cancel regressions.

Common misses

• Optimizes purely for nearest driver, missing the global opportunity-cost argument: assigning a driver to rider A can strand rider B, compounding into system-wide waste.
• Matches greedily per request rather than batching into windows, losing the near-global optimality that batching recovers at negligible added wait.
• Treats driver locations as exact rather than as estimates with age, so the design has no mechanism to score under uncertainty or handle stale GPS.
• Proposes naive A/B testing as the primary evaluation instrument, not recognizing that treatment and control units share the same marketplace supply and that simulation and switchback experiments are the honest alternatives.

The underlying concept

Matching markets punish myopia: each assignment consumes shared supply, so sequential greedy decisions compound into global waste — batching into windows recreates a small assignment problem where optimization is tractable, trading milliseconds of wait for system-wide efficiency. The engineering mirror: moving-object geospatial state wants cell-sharded in-memory indexes, and every distributed step must tolerate stale inputs and retries by design. Evaluation completes the lesson — when treatment units share a marketplace, simulation and switchbacks replace unit-level A/B as the honest instruments.

Source

Distilled Prep canon — curated from Uber's public work on dispatch, matching, and marketplace systems.

Source: Uber engineering blog

What the interviewer is probing

Whether the candidate recognizes this as an interference problem before choosing a method: driver-level randomization is the intuitive answer and it's wrong here, because treated and control drivers compete in one marketplace and the incentive changes behavior of both. Strong candidates frame the business decision first (expand or not, at what cost per incremental supply hour), pick a design that matches the interference structure, and are honest about what each design cannot identify.

What a strong answer covers

Must establish

• Frame expansion as a cost-effectiveness question: dollars per incremental supply-hour during peak, with downstream effects on rider wait times, completed trips, and driver retention as supporting metrics, and guardrails covering driver earnings dispersion, off-peak supply cannibalization, and rider price effects.
• Rule out naive designs by name: driver-level randomization suffers spillovers because treated and control drivers compete in the same marketplace, and pre/post comparison confounds seasonality and city trends.
• Choose a design that matches the interference structure: geo-randomization at city or zone-cluster level if enough units exist, switchbacks if effect onset and decay are fast, or synthetic control or diff-in-diffs against comparable cities when only one launch city exists, which is the realistic case here.

A strong answer adds

• Name anticipation and learning effects: announced start dates create pre-period contamination, and drivers take time to respond, so the design must define a burn-in period and a separate effect window.
• Measure incrementality by comparing supply-hours against the counterfactual, then decompose the source: new drivers activated versus existing drivers shifting hours from off-peak (cannibalization) versus hours shifted from neighboring zones (displacement). Only newly activated drivers are cleanly incremental.
• Close with a decision rule: expand if cost per incremental peak supply-hour beats alternative levers such as surge subsidies at comparable guardrail impact, validated across two or three structurally different cities before general rollout.

Common misses

• Proposes driver-level randomization without recognizing that control drivers face altered competition from treated drivers, inflating apparent treatment effects.
• Uses a pre/post comparison without accounting for seasonality or city-level trends.
• Measures that supply went up without decomposing incrementality into new activation, off-peak cannibalization, and geographic displacement.
• Does not frame the expansion decision as an economic cost-effectiveness trade-off against alternative supply levers.

The underlying concept

The Stable Unit Treatment Value Assumption (SUTVA) — that one unit's treatment doesn't affect another unit's outcome — fails almost everywhere in a marketplace, because participants compete for shared demand and supply. When SUTVA fails, the choice of randomization unit is the real design decision: you must randomize at the level that contains the interference (zones, cities, time blocks), even though this slashes your sample size and statistical power. That power-vs-validity trade-off, and the discipline of measuring incrementality against a counterfactual rather than a before/after, is the core of applied marketplace causal inference.

Source

Distilled Prep canon — curated from Uber's public work on marketplace experimentation and causally-informed optimization.

Source: Uber engineering blog

What the interviewer is probing

Whether the candidate can reason about cold-start under real operational pressure: transfer from similar cities, use of covariates that exist before launch (population, POI density, events), honest uncertainty communication, and a plan for graduating from priors to learned patterns. Strong candidates also ask what decision the forecast feeds — positioning tolerance determines how much error is acceptable.

What a strong answer covers

Must establish

• Clarify the decision consumer first: driver positioning tolerates different error than surge pricing, so the accuracy bar and the cost of over- vs under-forecasting must be established before any modeling choice is made.
• Day 0 requires transfer learning: pool data from demographically and geographically similar cities, and use features available pre-launch (census, POI density, weather, calendar) to drive the global model's predictions for the new city.
• Early weeks require hierarchical or Bayesian shrinkage so city-level estimates borrow strength from the global prior, with weight shifting toward local data as it accumulates; simple exponential blending is an acceptable pragmatic answer.

A strong answer adds

• Define an explicit graduation criterion for when the city transitions off the blended approach, for example when a local-only model beats the blended model on a rolling backtest.
• Name the feedback loop trap: positioning drivers changes observed demand, so the model observes fulfilled demand rather than true demand, which can corrupt training signal.
• Name the launch-period representativeness trap: early data is skewed by promotions and novelty effects and should not be treated as steady-state signal.

Common misses

• Does not frame the accuracy bar as a function of what decision the forecast feeds, treating all forecasting error as symmetric rather than asymmetric in cost.
• Jumps to modeling without identifying that pre-launch covariates (census, POI density, weather, calendar) exist and can drive the global model from day one.
• Does not acknowledge that observed trip data is censored: the model records fulfilled demand, not true demand, so naive training systematically underestimates demand in zones where supply was short.
• Treats launch-period data as representative steady-state signal without flagging promo and novelty distortions.

The underlying concept

Cold-start forecasting is fundamentally a bias-variance trade: with no local data, a global prior is high-bias but low-variance; local data is unbiased but high-variance early on. Hierarchical models formalize the blend, letting the data decide how quickly to trust local signal. The second deep idea is censored demand: a marketplace only observes demand it could serve, so naive training on fulfilled trips systematically underestimates demand exactly where supply was short — the places you most need to forecast well.

Source

Hand-written seed example in the style of Uber's marketplace forecasting posts.

Source: Uber engineering blog