Google

Google is unique on this site: their interview canon lives less in recent blog posts than in the foundational papers and systems their engineers built — the ideas that named entire categories. Interviews at Google skew toward generalist algorithmic and system-design rigor, but the company's published systems are the vocabulary the whole industry (and its interviewers) thinks in.

For software engineers, the classics are the syllabus: MapReduce (batch computation), Spanner (globally consistent databases), Borg (cluster scheduling, ancestor of Kubernetes), and Colossus/GFS (storage). For data scientists and MLEs, Google's experimentation culture (overlapping experiments, tiny-effect detection at enormous power) and ML production discipline (TFX, data validation) set the standards their interviews assume.

This page's canon questions are built on those enduring themes rather than the weekly feed — Google's research blog mostly wraps papers, so fresh material appears here more slowly than for other companies.

Their vocabulary

Borg: Google's cluster manager: schedules hundreds of thousands of jobs across machines, mixing latency-sensitive services with batch work. Kubernetes is its public descendant; scheduler questions assume its ideas.
Spanner: Google's globally distributed, externally consistent database — the system that showed strong consistency at planet scale is possible (with TrueTime's help). The reference point for global-consistency design questions.

Fresh from their blog

Derived from recent posts, newest first.

difficulty 3/5 ML system designML engData sciencegeospatialml-platformdata-qualityJul 2026

Your team needs a semantic segmentation model that distinguishes fine-grained land features — hedgerows, stone walls, isolated tree patches — in high-resolution satellite imagery. You have annotations covering only a few hundred square kilometers, but the target region is over 130,000 km². You have access to a vision-transformer backbone pre-trained on several hundred million satellite images from a broad global distribution.

How do you approach training and evaluation, how do you decide whether the pre-trained backbone is actually helping — and what are the ways this model will most likely fail silently when deployed across the full region?

Practice against the follow-up probes

How do you construct a validation set that gives you honest signal about generalization, given that your labeled data is geographically small and possibly clustered?
Pre-training was done on global imagery; your target is one country with a specific landscape character. What would make you distrust the feature representations the backbone learned?
After fine-tuning, you discover the model performs well on hedgerows near roads but poorly on field-interior hedgerows. What does that tell you about your annotation set, and how do you fix it?
How do you decide how much of the backbone to freeze vs. fine-tune, and what experiment tells you you've made the right call?
The model will be used to measure carbon stocks, so false negatives (missed hedgerows) have a real financial cost. How does that asymmetric cost change your modeling and evaluation choices?

Show answer guide

What the interviewer is probing

The question tests whether the candidate can reason about transfer learning not as a recipe but as a set of assumptions — about distribution shift, label density, and feature alignment — that must each be checked empirically. Metric judgment is probed through the carbon-accounting use case, which makes false-negative costs concrete. Causal reasoning about geographic clustering in the label set is the hardest dimension; strong candidates will recognize that spatially autocorrelated data requires spatial cross-validation, not random splits.

Strong answer outline

Why a pre-trained backbone helps: satellite imagery shares low-level texture features (vegetation spectral signatures, edge patterns, shadow geometry) across geographies; a backbone trained on hundreds of millions of global images will have learned these representations far better than a model trained from scratch on a few hundred km² of annotations. The fine-tuning hypothesis: the head and upper layers need to specialize to local landscape character; the lower layers may transfer almost directly.
Validation set design — the critical issue: spatial autocorrelation means that if you randomly split pixels or even patches, train and validation sets will be geographically adjacent and share context. Use spatial cross-validation: hold out geographically contiguous blocks (e.g., entire counties), not random patches. This gives honest generalization signal.
Checking that pre-training actually helps: ablation — train the same head from random init vs. from the pre-trained backbone, on the same labeled data, evaluated on the spatial hold-out. If the backbone doesn't improve sample efficiency (fewer labeled km² needed for the same performance), the distribution shift is too large and the representations don't transfer.
How much to freeze: start with the backbone frozen, establish a baseline, then progressively unfreeze upper layers and measure validation performance. The inflection point where unfreezing hurts (overfitting to the small labeled set) tells you where to stop. Monitor training vs. validation loss gap carefully.
Distribution shift failure modes in deployment:
Geographic bias in labels: if annotators covered accessible or visually distinct areas, the model learns those contexts. Field-interior hedgerows, less-managed boundaries, or features in remote uplands may be systematically missed.
Seasonal/illumination shift: fine-tuning imagery may be from one season; deployment imagery from another — vegetation appearance changes significantly.
Resolution or sensor shift: if inference imagery comes from a different satellite or processing pipeline, spectral statistics shift.
Rare but high-value features: stone walls may be underrepresented in training; the model's recall on them may be near zero without explicit oversampling or a weighted loss.
Asymmetric cost — false negatives: use a recall-weighted loss (e.g., focal loss or class-weighted cross-entropy biased toward the positive class), tune the classification threshold explicitly on the spatial validation set against a precision-recall curve, and report recall and false-negative rate per feature class as primary metrics, not aggregate pixel accuracy.
Wrong turns: using random pixel splits for validation (falsely optimistic generalization); treating aggregate IoU as the primary metric when the use case is carbon accounting (hides per-class recall differences); freezing the entire backbone without testing whether fine-tuning upper layers helps on this specific landscape character.

The underlying concept

Transfer learning from large foundation models works when two conditions hold: the source pre-training distribution shares low-level structure with the target task (so early-layer features transfer), and the target dataset, while small, is representative enough of the deployment distribution that fine-tuning the upper layers doesn't overfit to annotation artifacts. Geographic data violates the standard i.i.d. sampling assumption — nearby locations are correlated — so a random train/val split produces optimistically biased validation metrics. Spatial cross-validation, which holds out contiguous geographic blocks, is the correct analog to time-series cross-validation in temporal data: it simulates the actual generalization challenge, predicting on regions the model has never seen. Asymmetric misclassification costs should be encoded in both the loss function and the evaluation protocol from the start, not corrected post-hoc.

Source

Derived from From pixels to planning: Earth AI for nature restoration

difficulty 4/5 Product caseML engSoftware engexperimentationcausal-inferencenetworkingJul 2026

You work on a navigation platform and want to test whether rerouting a small fraction of vehicles away from congested segments improves city-wide driving speeds. The catch: the intervention isn't applied to individual users at random — it reshapes traffic conditions for everyone on the network, including drivers not using your app at all.

How do you design a valid experiment to measure the causal effect of this routing intervention? Walk through your choice of randomization unit, how you handle interference between treatment and control, and how you'd analyze the results.

Practice against the follow-up probes

Why can't you use a standard A/B test that randomly assigns individual trips to treatment vs. control?
You chose a switchback design alternating treatment and control by day. What are the main threats to validity with that choice, and how would you mitigate them?
How do you choose which road segments to target, and how does that selection affect what causal claim you can make?
You observe a 2% speed improvement on targeted segments but only 0.35% network-wide. What explains the gap, and which number should drive the business decision?
A critic argues that day-of-week confounding (e.g., treatment days happened to be less rainy) explains the result. How do you rule that out?

Show answer guide

What the interviewer is probing

This question tests whether the candidate immediately recognizes the network interference problem — the defining reason standard A/B testing fails here — and whether they can reason about the trade-offs of the available alternatives (geographic clustering, switchback/crossover, synthetic control). It also probes metric judgment: the distinction between local segment effects and network-wide effects is not cosmetic, and a strong candidate will articulate why only one of them represents the policy-relevant estimand. Finally, the follow-ups reveal causal discipline — can they enumerate the confounders a switchback design is exposed to and propose concrete countermeasures?

Strong answer outline

Why standard A/B fails: routing changes are not independent across users — diverting user A changes congestion for user B, so individual-level randomization produces SUTVA violations everywhere. Treatment and control units interfere, making estimated effects meaningless.
Randomization unit options and their costs:
Geographic clustering (randomize by city or district): eliminates within-city interference but requires many cities for power; spillover at cluster boundaries remains.
Switchback / crossover design (alternate treatment and control over time across the whole city): removes geographic spillover by giving the entire network the same treatment state at any moment. Cost: assumes the city reaches a new equilibrium quickly, and that consecutive-day effects don't bleed over (carryover bias).
Synthetic control (match treated cities to untreated ones): useful for city-level rollouts but requires rich pre-treatment data and assumes no network ties between cities.
Segment selection is a design choice, not a tuning knob: selecting ~100 historically congested segments creates a well-defined estimand (effect on those bottlenecks) but limits external validity to similar bottleneck types. Pre-registration matters here to prevent post-hoc fishing.
Carryover bias in switchback: traffic patterns have memory (rush hours, weekly seasonality). Mitigation: enforce washout periods between arms, balance arm assignment by day-of-week, and model residual autocorrelation explicitly in the analysis.
Analysis with hierarchical Bayesian model: pools information across cities and hours while allowing city-level heterogeneity; appropriate when per-city power is limited. Key output is a posterior distribution over the effect, not just a point estimate — which forces honest uncertainty communication.
Interpreting the two effect sizes: the ~2% targeted-segment effect measures local decongestion; the ~0.35% network-wide effect is the policy-relevant number because it captures both the gains on cleared segments and the costs imposed on absorbing segments. Reporting only the larger local number would overstate societal benefit.
Common wrong turns: claiming individual-level causal effects from a city-level design; ignoring that non-app users benefit (which is actually an argument for the intervention, not against measurement); conflating statistical significance with practical significance at 0.35%.

The underlying concept

Standard A/B testing rests on the Stable Unit Treatment Value Assumption (SUTVA): one unit's treatment does not affect another's outcome. Traffic networks violate this structurally — a rerouted vehicle changes travel times for every other vehicle on its new and old path, a phenomenon called interference or spillover. When interference is global (the whole network is one connected system), the only valid randomization units are ones that contain the interference: entire cities, or time periods in a switchback design. Switchback designs trade spatial isolation for temporal isolation, measuring the city's response to switching the entire system between states, but they introduce carryover risk whenever the system's equilibrium lag exceeds the switching interval. The causal estimand must be defined at the level of the randomization unit — network-wide average speed, not individual trip time — or the estimate is not causally identified.

Source

Derived from The power of collaboration: How we can reduce traffic congestion

difficulty 4/5 Systems designSoftware engML engcachingperformanceml-platformJul 2026

You run a globally distributed, Spanner-like database serving billions of requests per second — replicas on several continents, and a cache miss can mean an expensive cross-shard or cross-region read. Memory is billed by the gigabyte-hour, making it a metered utility rather than a sunk cost. Your current cache is statically sized to handle peak load, which means you're paying for idle memory most of the day.

Design a cache management system that treats memory as a variable cost and dynamically adjusts what it holds — and how long it holds it — to minimize total cost of ownership without unacceptably degrading I/O performance.

Start by explaining how you'd frame the eviction decision, then walk through the architecture from prediction to eviction, and finally tell me how you'd validate that the system is actually saving money and not silently destroying tail latency.

Practice against the follow-up probes

You framed this as an optimization problem — what's the objective function, exactly, and what are its inputs? Walk me through the math.
You're assigning time-to-live values per cached page. How do you learn good TTLs, and what features do you train on?
Your TTL model runs on every cache hit at billions of requests per second. What happens if the model is even slightly too slow?
TTL eviction and capacity-based eviction (e.g., LRU) are now both active. How do you reason about their interaction, and which one takes precedence under what conditions?
Memory pricing shifts, or a new workload type appears with very different miss costs. How does the system adapt without a manual re-deployment?

Show answer guide

What the interviewer is probing

This question probes whether the candidate can translate an economic constraint (pay-per-byte-hour) into an algorithmic design, rather than treating resource allocation as a static engineering parameter. It tests architecture judgment — separating the TTL assignment problem from the capacity management problem — and forces the candidate to reason about the feedback loops between a prediction model and the system it controls. The validation portion surfaces metric judgment: distinguishing between cost savings and latency degradation, and knowing which cache miss increases are acceptable.

Strong answer outline

Frame the cost model first. Total cost = (memory cost × bytes cached × time held) + (miss penalty × miss rate). The insight is that holding a page whose next access is far away costs more in memory rental than the miss penalty you'd pay to re-fetch it. This reframes eviction as: "when should I stop renting this page's slot?"
The ski-rental analogy. Each cached page faces the same rent-vs-buy decision under uncertainty about future access. The break-even rule: keep the page as long as its accrued memory cost is below the miss penalty; evict when it crosses. A randomized variant reduces expected cost versus the adversarial worst case.
TTL assignment via a lightweight model. Features: data size (miss penalty scales with it), operation type (read vs. scan), historical inter-arrival time for this page, relative miss cost vs. memory cost ratio. Candidate should reach for a shallow decision tree or small lookup table — not a neural net — because inference runs on every cache hit in a hot path. Wrong turn: proposing a complex model without noting the latency budget.
Capacity eviction as a backstop. When the cache fills, fall back to a generalized LRU (or GDSF to handle variable-size pages). TTL-based eviction shrinks the working set first; LRU handles physical overflow. These are complementary, not competing.
Validation strategy. Offline: replay cache traces with the new policy vs. fixed-size baseline; measure integrated memory-time cost and miss rate. Distinguish miss rate by miss cost — a high miss rate on cheap-to-fetch rows is acceptable; misses on expensive cross-shard reads are not. Online: shadow the new policy against production traffic before cutover; track p99 latency, actual I/O cost per request, and memory footprint over time. Alert on tail-latency degradation, not just aggregate miss rate. The critical wrong turn: using miss rate alone as the health signal — a cost-aware system deliberately increases misses on cheap data.
Adaptation. TTL model should be retrained on rolling windows of production access logs. Feature drift (e.g., a new query pattern) shows up as systematic TTL misprediction; detect this by monitoring model-predicted TTL vs. actual next-access gap for sampled pages.

The underlying concept

Static cache sizing is a peak-provisioning problem: you allocate for the worst hour and pay for all the idle hours in between. When memory is metered, this becomes a literal dollar cost rather than an amortized hardware cost, which makes the economics of dynamic sizing much more attractive. The ski rental problem is the canonical model for this class of decision: you must commit to a recurring cost (renting memory) or a one-time cost (paying the miss penalty) without knowing how long you'll need the resource. The break-even algorithm — rent until your total rental equals the buy price, then buy — is worst-case optimal within a factor of 2. In practice, access patterns are predictable enough that a learned model can beat the adversarial bound significantly. The deeper structural insight is that TTL-based eviction and capacity-based eviction solve different subproblems — TTL controls the holding duration under normal load, and LRU/GDSF controls behavior when the cache fills — and keeping them cleanly separated makes both easier to reason about and tune.

Source

Derived from Optimizing cloud economics with linear elastic caching

The canon

Curated questions on this company's enduring themes.

difficulty 4/5 Product caseData scienceexperimentationab-testingJul 2026

At a product with a billion daily users, nearly every experiment is statistically significant — a 0.02% change reaches p < 0.001. Design the decision framework that determines which changes actually launch, and how you would run hundreds of experiments on the same surface concurrently.

Practice against the follow-up probes

If everything is significant, what replaces significance as the launch bar?
How do you compare a +0.02% revenue change against a -0.01% latency guardrail move?
Hundreds of concurrent experiments on one surface: how do you avoid collisions, and when do interactions actually matter?
How do you control the false-launch rate across thousands of experiments a year?
A tiny effect is significant in aggregate but you suspect it's driven by one country. How do you investigate without p-hacking?

Show answer guide

What the interviewer is probing

Whether the candidate understands that at extreme power the binary significance question dissolves and the real questions become effect size, practical value, and portfolio-level error control. Also probing knowledge of overlapping-experiment infrastructure — layered traffic allocation — which Google pioneered and which any candidate claiming experimentation depth should be able to reconstruct from first principles.

Strong answer outline

Replace "significant?" with "big enough to matter?": pre-registered practical-significance thresholds per metric (minimum launch-worthy effect), confidence intervals reported against those thresholds rather than zero.
Value framework: convert core metrics to a common currency (long-term value per user) with explicit exchange rates set by leadership — e.g., how much revenue a millisecond of latency is worth — so mixed-sign results are decidable rather than debated ad hoc.
Long-term validity: tiny short-term gains can be novelty or cannibalization; use long-running holdbacks to measure cumulative and lagged effects, and validate proxy metrics against them.
Concurrency: layered/orthogonal experiment infrastructure — divide the system into layers (e.g., UI, ranking, ads) where experiments within a layer split traffic exclusively but layers overlap freely via independent hashing; run explicit interaction tests only for suspected-conflicting pairs.
Portfolio error control: with thousands of tests, control the expected number of false launches (higher evidence bars for launch-gating metrics, replication runs for surprising wins, holdback verification post-launch).
Segmentation discipline: pre-registered segment analyses or hierarchical shrinkage estimates; exploratory segment findings get confirmed in a fresh experiment, never launched from the discovery data.

The underlying concept

Statistical significance answers "is the effect exactly zero?" — a question nobody asked. With enormous samples, standard errors collapse and every real-but-trivial effect clears p-thresholds, so decision theory must take over: minimum effects of interest, explicit trade-off prices between metrics, and error control at the portfolio level (false launches, not false positives). Overlapping-layer designs solve the concurrency problem by making experiment assignments statistically independent across layers, spending scarce exclusive traffic only where interference is real.

Source

Distilled Prep canon — built on Google's published experimentation methodology (overlapping experiments, long-term holdbacks).

Source: Distilled Prep canon (curated)

difficulty 4/5 ML system designML engml-platformml-monitoringdata-qualityJul 2026

Design the production ML pipeline for a model that retrains continuously on fresh data — covering validation, training, deployment, and the guarantee that a bad data day never silently degrades the serving model.

Practice against the follow-up probes

What specifically can go wrong with the data between yesterday and today, and how does each failure manifest?
What does automated data validation check, and against what reference?
Training/serving skew: where does it creep in and how is it prevented structurally?
What gates stand between a freshly trained model and production traffic?
When the model degrades in production anyway, how do you localize cause quickly?

Show answer guide

What the interviewer is probing

Production-ML maturity: the understanding that most real-world model failures are data failures — schema drift, distribution shift, broken upstream joins, feature pipeline bugs — and that reliability comes from treating data like code: schemas, validation, versioning, and gated promotion. The TFX worldview, reconstructable from first principles.

Strong answer outline

Failure taxonomy: schema changes (renamed/dropped fields, type changes), distribution shift (gradual drift vs. sudden breakage), missing partitions, upstream logic changes silently redefining a feature, and label problems (delay, leakage).
Data validation stage: maintain a versioned schema + expected statistics per feature (ranges, missingness, distributions); validate each training batch against it — hard-fail on schema violations, alert on statistical anomalies (drift scores vs. a reference window); quarantine bad batches rather than train on them.
Skew prevention structurally: single feature-definition source compiled to both training and serving (shared transforms / feature store); continuously compare serving-time feature distributions against training distributions — skew alarms are among the highest value monitors.
Training pipeline: versioned data snapshots + versioned code + config = reproducible artifacts; evaluate on holdout and on sliced metrics (segments where regressions hide) against the current production model as baseline.
Promotion gates: offline eval thresholds → canary serving on small traffic with online guardrails → gradual ramp with automatic rollback; the previous model stays warm for instant reversion.
Diagnosis loop: when production quality dips, the versioned lineage (which data, which code, which model) plus feature-distribution monitors localize whether it's data, model, or world; without lineage, every incident is archaeology.

The underlying concept

Continuous training turns ML into a data-processing system whose correctness depends on inputs no one reviews — so the engineering answer is to give data the same discipline code has: declared schemas, automated validation, version control, and staged rollout. The two structural insights: skew is prevented by construction (one definition, two consumers), not by vigilance; and models should be promoted like binaries — evaluated against the incumbent, canaried, and reversible — because "newer" is not "better" when the data decides.

Source

Distilled Prep canon — built on the themes of Google's TFX and ML production-systems publications.

Source: Distilled Prep canon (curated)

difficulty 5/5 Systems designSoftware engdistributed-systemsdatabasesJul 2026

Design a globally replicated database for advertiser account budgets: a campaign must never spend past its budget, updates can arrive on any continent, and ad-serving decisions read budgets millions of times per second worldwide.

Practice against the follow-up probes

Strong consistency across continents costs round trips. Where exactly do you pay it, and where do you refuse to?
How can ad servers make spend decisions at local latency without letting global spend exceed budget?
What does a consensus protocol buy you here, and per what unit would you run it?
Clocks: why do global transactions care about time, and what breaks with ordinary NTP?
An entire region is partitioned away mid-campaign. What happens to spending?

Show answer guide

What the interviewer is probing

Whether the candidate can localize the strong-consistency requirement (budget commitment) and engineer around paying cross-continent latency on the hot path — the budget-reservation/quota-lease pattern. Also probing real distributed-systems literacy: consensus per shard, read-path options (leases, bounded staleness), and why globally ordered transactions need bounded clock uncertainty.

Strong answer outline

Split the problem: the invariant (total spend ≤ budget) needs serialized commitment; the decisions (serve this ad?) need microsecond reads. Never put the second on the first's path.
Budget ledger: shard by account/campaign; each shard is a consensus-replicated state machine (Paxos/Raft) across regions — writes commit with cross-region quorum (pay the ~100 ms once per budget mutation, which is rare traffic).
Local spending at global safety: regions lease budget slices from the ledger (e.g., region takes a 5% tranche); ad servers decrement local tranches at memory speed and renew leases asynchronously. Overspend is bounded by outstanding tranche size — pick tranche sizes to trade slight budget under-utilization against overspend risk.
Reads: serving reads hit local replicas/leases (bounded staleness by construction); reporting reads can request linearizable reads through the leader when exactness matters.
Time: to give external consistency (transactions ordered as observed globally) you need bounded clock uncertainty — the TrueTime idea: expose uncertainty intervals and wait them out on commit; with plain NTP you either wait too long or violate ordering under skew.
Partition behavior: a cut-off region can spend only its already-leased tranches, then fails closed (stops spending) — bounded damage, no global invariant violation; leases expire and return to the ledger.
State the CAP position explicitly: choose consistency for the ledger; engineered availability comes from leases, not from weakening the invariant.

The underlying concept

Global strong consistency is affordable when you stop buying it wholesale: identify the single invariant that needs serialization, run consensus for exactly that, and convert the hot path into locally served, pre-authorized capacity — the quota-lease/escrow pattern that turns a global constraint into bounded local ones. The Spanner lesson completes it: global transaction ordering is fundamentally a clock problem, solved not by perfect clocks but by known-bounded uncertainty plus commit waits.

Source

Distilled Prep canon — built on the themes of Google's Spanner and planet-scale infrastructure work.

Source: Distilled Prep canon (curated)

difficulty 3/5 Product caseData sciencesearch-rankingexperimentationml-monitoringJul 2026

A search ranking change increases result click-through rate, but query reformulations and back-button returns to the results page also rise. Interpret the experiment and recommend a decision.

Practice against the follow-up probes

What story reconciles more clicking with more re-searching?
Which metrics distinguish "attractive results" from "useful results"?
How would you measure whether users actually found what they needed?
What role do time-based signals (dwell time, time-to-success) play, and what are their pitfalls?
If the effect differs by query type, how does that change the launch decision?

Show answer guide

What the interviewer is probing

Metric literacy in search: CTR measures the promise of a result (title/snippet attractiveness), not its delivery. Rising reformulations and pogo-sticking are classic signals that clicks are failing to satisfy. Strong candidates assemble a success-oriented metric story (abandonment done right, long clicks, session-level success) and slice by query intent before deciding.

Strong answer outline

The reconciling story: the change likely surfaces clickable results — compelling titles/snippets — that under-deliver; users click, bounce back (pogo-stick), and re-query. CTR up + reformulation up = attractiveness up, relevance flat or down.
Better metric set: long clicks (dwell above threshold) vs. short clicks; pogo-stick rate; session-level success (did the session end in a satisfied state?); time-to-result; and good abandonment (query answered on the results page itself) separated from bad abandonment.
Diagnose by intent: navigational queries (one right answer), informational, transactional — a change can help one class and harm another; the aggregate hides it. Also slice by position: is added CTR coming from top positions or deep exploration?
Careful with dwell: long dwell can mean satisfaction or struggle; calibrate time signals per query class against explicit-feedback panels or human relevance ratings.
Recommendation shape: as described, the evidence pattern points against launch despite CTR; either iterate (snippet honesty, relevance) or launch only for the query classes where session success genuinely improved.
Prevention: make session-success and pogo-stick guardrail metrics so attractiveness-only wins can't clear the launch bar again.

The underlying concept

Every proxy metric encodes an assumption about what it proxies for; CTR assumes clicking predicts satisfaction, and ranking changes can break exactly that link by optimizing the promise rather than the payoff. Search measurement matured by moving from action counts to session-level success models — the unit of value is the user's task, not the click. The generalizable habit: when two proxies disagree (clicks up, re-queries up), the disagreement itself is the finding, and the tiebreaker is the metric closest to the user's true goal.

Source

Distilled Prep canon — built on Google's published work on search quality measurement.

Source: Distilled Prep canon (curated)

difficulty 4/5 Systems designSoftware enginfrastructuredistributed-systemsscalabilityJul 2026

Design a cluster scheduler that places both latency-critical services and batch jobs across hundreds of thousands of machines, maximizing utilization without harming the critical services.

Practice against the follow-up probes

Why does mixing service and batch work on the same machines matter economically, and what makes it dangerous?
What does a job specification need to contain for the scheduler to do its job?
Placement is a constrained optimization. What are the constraints and the objective, and how do you keep scheduling latency low at this scale?
A top-tier service suddenly needs capacity in a full cluster. What happens, step by step?
How do you handle machine failures, maintenance, and the scheduler itself failing?

Show answer guide

What the interviewer is probing

Understanding of the core datacenter economics — services reserve for peak and idle most capacity, so reclaiming that idle capacity for preemptible batch work is worth billions — plus the mechanisms that make sharing safe: priorities/tiers, preemption, resource isolation, and overcommit informed by actual usage vs. reservations. This is Borg territory; Kubernetes-only intuition covers maybe half of it.

Strong answer outline

Why mix: dedicated pools idle at 20-40% utilization because services size for peak; co-locating preemptible batch in the trough reclaims it. The danger is interference — batch stealing CPU cache, memory bandwidth, or IO from latency-critical work.
Job spec: resource requests (CPU, RAM, disk, accelerators), priority tier (production vs. best-effort), constraints (region, hardware, anti-affinity), and disruption tolerance. Quota is enforced at admission, priority at runtime.
Scheduling loop: feasibility filtering (which machines can host it) then scoring (bin-packing for utilization, spreading for failure domains, data locality); at this scale use equivalence classes / cached scores and randomized sampling of candidate machines rather than scoring the whole fleet; shard or replicate schedulers with optimistic concurrency on placement commits.
Priority mechanics: the top-tier service preempts best-effort tasks — scheduler evicts enough batch (respecting disruption budgets), reclaims resources, places the service; batch tasks are checkpointed or simply rescheduled — their contract is "will finish eventually."
Overcommit safely: schedule against predicted actual usage rather than reservations for lower tiers, with node-level isolation (cgroups-style) and a killer that reclaims from best-effort first when a node runs hot.
Reliability: scheduler state rebuilt from node agents (nodes are the truth); running workloads unaffected by scheduler outages; masters consensus-replicated per cell; failure domains and maintenance handled via eviction with notice.

The underlying concept

Cluster scheduling is economically a statistical multiplexing problem: peak-reserved capacity is mostly air, and tiered priorities plus preemption convert that air into throughput while preserving the latency tier's guarantees. Technically it's constrained optimization under a latency budget — solved not by optimality but by good-enough placement found fast (sampling, caching, sharding). The deepest design rule: keep the data plane independent of the control plane, so the scheduler's own failures never take down what it scheduled.

Source

Distilled Prep canon — built on the themes of Google's Borg and cluster-management publications.

Source: Distilled Prep canon (curated)