MLOps Interview Questions

"MLOps is the set of practices for getting models into production reliably and keeping them working once they're there. It borrows a lot from DevOps: version control, automated tests, CI/CD, monitoring. The difference is that in an ML system the behaviour depends on three things, not one: the code, the data it was trained on, and the trained model itself. So I have to version all three. The other big difference is that a model can get worse without anyone touching it, because the world changes and the incoming data drifts away from the training data. That's why MLOps adds pieces a normal service doesn't have: experiment tracking, a model registry, pipelines that retrain on their own, and monitoring of the data and predictions, not just CPU and error rates."

"My approach is to make the production path the easy path. I give data scientists a project template that already logs to the tracking server, saves the full pipeline and has a serving wrapper, and I join early design chats instead of receiving a notebook at the end. One disagreement I remember: a data scientist wanted to use a feature built from a slow reporting query that ran once a day, and the model was meant to score in real time. She saw a clear lift offline. I worried the feature would be up to a day stale at serving, which her offline test didn't reflect. We settled it with data: I rebuilt the training set with the feature lagged the way it would really be served. Most of the lift survived, so we kept it, and we both started testing features under serving conditions by default."

"I start from when the input exists and how fresh the answer has to be. If I can know the inputs ahead of time, like scoring every customer for churn once a night, batch is my default. A scheduled job scores everything, writes to a table, and the app just reads it. It's cheap, easy to debug and there's no latency pressure. I need a real-time endpoint when the input only exists at the moment of the request, like a fraud check on a card payment or ranking search results for the query someone just typed. That brings a latency budget, autoscaling and an on-call burden, so I only pay that cost when the product needs it. There's also a middle path: scoring events from a stream as they arrive, or precomputing heavy features in batch and doing a light online lookup."

"The weights on their own aren't enough. First, the preprocessing: scaling, encoders, tokenisers, whatever turned raw input into features during training. I save those together with the model as one pipeline object, so serving can't quietly do it differently. Second, the environment. Formats like pickle or joblib can break or behave differently if the library version changes, so I pin the exact versions and usually build a container image from them. Third, a contract: the input schema with names and types, the output format, and metadata like the model version, which data snapshot and code commit produced it, and its evaluation metrics. Tools like MLflow bundle a lot of this, with a signature and the environment file, but the principle is the same whatever tool I use: someone should be able to load and call it without asking me anything."

"I load the model at startup, because loading it per request would add seconds of latency. Input goes through a typed schema, so a missing field or a string where I expect a number gets a clear 422 error instead of a strange prediction. The handler builds a one-row frame with the same column names used in training and returns the score with the model version, which I read from the environment so every prediction can be traced back. Before real traffic I'd add a readiness check that runs one test prediction, structured logs of inputs and predictions for monitoring, a request timeout, and a load test to find how many workers one instance can handle. If throughput mattered a lot, I'd look at batching requests together."

"I'd start by agreeing with the data scientist on what the gain is worth, because a small offline lift may not show up in any business metric. Then I'd be clear the latency budget isn't mine to bend; it's there because a slow response hurts the product. But I don't want to just say no. We'd profile the model and try the cheap wins first: an optimised runtime, quantisation, trimming features that are expensive to fetch but add little. If it's still too slow, there are design options. Distil it into a smaller model, precompute its scores in batch where inputs allow, or use a two-stage setup where a fast model handles most requests and the big one only scores the uncertain cases. Whatever we try, I'd test it in shadow first to check real latency before any user sees it."

"First I'd trace one request end to end and split the time: network, fetching features, preprocessing, the model call and time spent waiting in a queue. The tail is often not the model at all. Common culprits are a feature store call that sometimes times out, a new instance serving its first requests before it's warmed up, or a queue building when traffic bursts faster than autoscaling reacts. Then I'd fix what the trace shows. For cold starts, send warm-up requests before marking the instance ready. For slow lookups, fetch features in parallel, cache hot entities and set tight timeouts with a fallback. If the model itself is slow, I'd try exporting to an optimised runtime, quantising, or distilling to a smaller model, and check with the data scientist that accuracy holds. For bursts, keep some headroom and scale on queue depth rather than CPU."

"First I'd make the spend visible: tag everything by team, model and job, and work out cost per training run and per thousand predictions. Usually a few things dominate. Then I look for waste, which is often the biggest win: GPUs sitting idle, dev notebooks left running, endpoints with no traffic, and jobs showing low GPU utilisation because they're starved by data loading. For training, I'd move fault-tolerant jobs to spot or preemptible capacity with regular checkpoints, use mixed precision, and cut hyperparameter sweeps with early stopping. For serving, I'd check whether small models even need a GPU, add batching, try quantised or distilled models with the data scientists, and set autoscaling so endpoints scale down at quiet times. I'd track each change against quality and latency, so savings don't quietly cost accuracy."

"Training-serving skew is when the features a model gets in production don't match what it saw in training, even for the same real-world input. The classic cause is two implementations: a data scientist computes a feature in SQL for training, and an engineer rewrites it in the service code. Small differences creep in, like a different time window, a timezone, or how nulls are filled. To prevent it, I make sure each feature has one definition that both training and serving use, either through a feature store or a shared library. Even better, I log the exact features the service used at prediction time and build the next training set from those logs, so training and serving literally share the data. Then I still monitor by comparing served feature distributions with training ones, as a safety net."

"A feature store solves two problems. Teams keep rebuilding the same features, like a customer's spend over the last 30 days, in slightly different ways. And a feature gets computed one way for training and another way in the live service, which causes skew. A feature store gives each feature one definition, registered once and reused. It usually has two storage layers because the access patterns are opposite. Training needs history: what was this customer's value at every point in the past, for millions of rows, and a warehouse or data lake is good at that. Serving needs the latest value for one customer in a few milliseconds, and a key-value store like Redis is good at that. The feature store keeps both in sync from the same definition, so the model sees the same logic in both places."

"When I build training data, each label has a timestamp, the moment I would have made a prediction. A point-in-time join attaches, for each of those events, the feature values as they were just before that moment. If I instead join today's feature values, I leak the future. Say a feature is number of chargebacks on the account. Today's value already includes the chargeback from the fraud I'm trying to predict, so the model learns a shortcut that won't exist live. Offline metrics look amazing and production is disappointing. Feature stores do this join for me when I give them entity IDs and event timestamps. In pandas I can do it with merge_asof, looking backward only. I also account for when a value actually became available, because a feature computed by a nightly job isn't usable until that job finishes."

"Tracking is the lab notebook. Every training run logs its parameters, metrics, artifacts like the model file and plots, and ideally the code commit and data version. That lets me compare dozens of runs and see what actually helped. The registry is the catalogue of models that matter. I register a model under a name, and each time I register a new one it gets the next version number, linked to the run that produced it, so I can always trace a production model back to its exact training run. Then I mark which version is approved. Older setups used fixed stages like Staging and Production. Recent MLflow versions push you towards aliases, like champion and challenger, plus tags. My serving code loads the model by alias, so promoting or rolling back is just moving the alias, with no code change."

"Data is too big for Git, so I version it by reference. There are a few ways I've seen work. One is a data versioning tool like DVC, where the files sit in object storage and Git only tracks small pointer files with content hashes, so checking out a commit and pulling gets you the matching data. Another is a table format with snapshots, like Delta Lake or Iceberg, where I can read a table as of a specific version or timestamp. The simplest is immutable, dated folders that nobody overwrites. Whatever the method, the key part is linking it: each training run logs the data version or snapshot ID next to the code commit and parameters. Then any model in the registry can be traced to exactly what it learned from. I also watch retention, because a snapshot that gets cleaned up breaks that link."

"There are several sources. Random seeds for weight initialisation, shuffling and the train-test split, if they're not fixed. Data loading with several workers, where each worker needs its own seeded random state for augmentation. On GPUs, some operations use parallel reductions where the order of floating point additions varies, so results differ in the last digits and that can snowball over a long run. And library or driver versions, which can change numerics. To tighten it, I set seeds for every library involved, fix the split by saving it or hashing IDs, pin the whole environment in a container, record the data snapshot, and use the framework's deterministic settings, knowing they can slow training. In practice I aim for bit-exact rebuilds only where an audit requires it. Otherwise I define an acceptable tolerance on metrics and check reruns land inside it."

"In normal software, CI tests code and CD ships it. In ML there are two things being delivered: the training pipeline, which is code, and the models that pipeline keeps producing. CI covers the code side. I run unit tests on feature transformations, check schemas, and do a fast training run on a small sample to prove the pipeline works end to end. CD deploys that pipeline to the production orchestrator. Continuous training is the extra loop: the deployed pipeline reruns on fresh data, on a schedule or when monitoring triggers it, and outputs a candidate model. That candidate goes through its own gates: validation of the new data, evaluation against the current model on a fixed test set and on important segments, and checks on latency and size. Only if those pass is it registered and rolled out."

"I think in three layers. Code tests are the normal ones: feature functions tested with nulls, empty frames, unexpected categories and extreme values. Data tests run before training: is the schema what I expect, are the row counts and null rates in a sane range, is the data fresh, did a column suddenly become constant. These catch the upstream breakages that cause most real failures. Model tests run after training: a quick end-to-end run on a small sample in CI to prove the pipeline works, then on the real run a minimum metric bar, comparison against the current production model, metrics for key segments so an average doesn't hide a regression, and a few behaviour checks, like a prediction that shouldn't change when an irrelevant field changes. I also test that the saved model loads and gives identical outputs in the serving environment."

"The gate takes metrics for the candidate and the current champion, both computed on the same frozen test set, because comparing numbers from different data means nothing. It requires a real gain overall, not just noise, so there's a minimum improvement. Then it checks each important segment, because a model can improve the average while getting worse for, say, new users. It enforces a latency limit so an accurate but slow model can't slip through. It returns the reasons rather than a bare false, so when a pipeline blocks a model the team can see why straight away. In the pipeline, a pass registers the model and moves it to a canary, and a fail stops the run and alerts the owner."

"For fraud, labels arrive late, because chargebacks can take weeks, so I only train on transactions old enough to have settled labels. A trigger starts the run: a schedule, a drift alert, or a drop in measured precision. Step one validates the new data: schema, volumes, fraud rate in a sane range. If that fails, the pipeline stops and alerts. Step two builds a point-in-time correct training set from the feature store and trains, logging everything to the tracking server. Step three evaluates against the current champion on a fixed test set and on the most recent labelled window, including key segments and latency. If it passes, it's registered and deployed in shadow, scoring live traffic without acting. After a few days of comparing, it goes to a small canary, then full traffic, with automatic rollback if alert rates or latency go out of bounds."

"In my last role, a pricing model was retrained every month by one data scientist running notebooks on her laptop, then emailing a file to the backend team. Releases took about a week and once went out with the wrong preprocessing. I didn't try to build everything at once. First I moved the notebook logic into a versioned training script and logged every run to MLflow, so we could reproduce a model. Next I registered models and changed the service to load by alias, which removed the emailed files. Then I added an orchestrated monthly pipeline with data checks and a comparison against the current model. A release went from about a week to under a day, and nobody had to remember steps. The lesson was to involve the data scientist early; she found issues in my version of her feature code that I'd have missed."

"Without labels I watch everything around the model. On the input side, data quality first: missing values, new categories, values out of range, broken pipelines. Then drift: how far each important feature's distribution has moved from the training data. On the output side, I track the distribution of predictions. If a model that usually flags two in a hundred suddenly flags ten, something changed, even if I can't say it's wrong yet. Then I look for early proxies from the business, like how many flagged cases a review team confirms, or click-through for a ranking model. And I log every prediction with an ID, so when the real outcomes arrive weeks later I join them back and compute the true metrics by cohort. Drift alone doesn't prove the model is worse, so I treat it as a reason to look, not an automatic verdict."

"I take the bin edges from the reference data, usually the training set, using quantiles so each bin holds about the same share. I clip the current data into the reference range, so values beyond it land in the end bins rather than being dropped. Then I compute the share of each dataset in each bin, floor them with a tiny number so an empty bin doesn't give a log of zero, and sum the difference times the log ratio. A common rule of thumb is that below 0.1 is little change, 0.1 to 0.25 is worth a look, and above 0.25 is a big shift. I treat those as starting points and tune thresholds per feature, because with huge samples tiny, harmless shifts can still register, and I care more about drift in the features the model relies on most."

"I'd build it in four layers, because each one catches a different failure. The service layer is the same as any API: requests per second, error rate, latency at the median and the tail, and CPU, memory or GPU use. That tells me it's up. The data layer checks what's coming in: schema violations, missing value rates and drift on the important features compared with training. The model layer tracks the distribution of predictions and, once true outcomes arrive, accuracy or precision and recall, split by key segments. The business layer shows the thing the model is supposed to improve, like fraud losses caught or conversion. Every chart shows which model version was live, so I can connect a change in the numbers with a deployment. Alerts go on a few of these, not all, to avoid noise."

"It depends on how fast the domain moves and how quickly I get labels back. A schedule is the simplest: retrain weekly or monthly, every run goes through the same evaluation gate, and it's easy to plan. It works well when change is gradual. A drift trigger reacts faster when inputs change, but drift isn't the same as a worse model, so I'd use it to start an evaluation, not to push a new model blindly. A performance trigger is the most direct signal, but it only works when labels come back reasonably fast. In practice I blend them: a schedule as the baseline, plus triggers on big drift or a measured drop that can start a run early. Whatever starts it, the new model still has to beat the current one before it ships."

"An alert people ignore is worse than none, because the real one gets missed too. First I'd look at the history: which features fire, by how much, and whether any alert ever lined up with a real drop in model quality. Usually a few things cause most of the noise, like a statistical test that flags tiny shifts because traffic is huge, or a feature with a normal weekly pattern compared against a flat baseline. Then I'd retune. Thresholds per feature, stricter for the features the model relies on most. A baseline that respects seasonality, like the same weekday last month. And require a shift to last before alerting. Finally I'd split the outputs: page someone only for drift on important features or a drop in prediction quality, and send the rest to a weekly report that someone owns and reviews."

"At my last company we had a demand forecasting model feeding stock orders. Planners told us its forecasts for one product category had looked odd for about two weeks. We had service monitoring, but nothing on the data. When I dug in, an upstream team had changed a category code, so a whole group of products started arriving with an unknown category. Our pipeline filled it with a default, and the model treated them like a much smaller category. I fixed the mapping, backfilled the features and retrained, and forecasts came back in line. The bigger change was stopping it happening silently again. I added data validation before both training and scoring, alerting on unseen categories and jumps in default values, and we agreed a schema contract with the upstream team so changes get announced first."

"They answer different questions, so I often use them in order. In shadow mode the new model gets a copy of live requests, but its predictions are only logged. Users never see them. That tells me it handles real traffic, what its latency is, and how often it disagrees with the current model, with no user risk. It can't tell me the business effect, because nothing acts on its output. A canary sends a small slice of real traffic to the new model and watches errors, latency and key metrics. It's a safety step: if something is off, only a few users are hit and I roll back. An A/B test is an experiment: a random split, a chosen metric, a planned sample size and duration. That's what tells me the new model actually improves the outcome, beyond just not breaking."

"If the timing lines up with the release, I roll back first and investigate second. The previous version should still be deployable in one step, like moving the registry alias back, so this takes minutes. I'd check the metric starts recovering, and post in the incident channel what we saw and what we did. Then I dig in: compare the two versions' predictions on the same traffic, look at which segments moved, and check whether the new model got different inputs than it was tested on, like a feature that's null in production. It's also worth confirming it really was the model and not another release at the same time. Afterwards I'd ask why the rollout let this reach everyone. The fix is usually a canary stage with automatic rollback on that metric, so next time it hits a small slice."

"A classic model does one forward pass per request, cheap and predictable. An LLM generates one token at a time, so a long answer takes many passes and latency depends on how much it writes. There are two phases: prefill, which processes the whole prompt at once and is compute heavy, and decoding, which is usually limited by memory bandwidth. Memory is the big constraint. Besides the weights, each request keeps a KV cache, the stored attention keys and values for every token so far, which grows with context length, so long prompts and many concurrent users can run out of GPU memory. That's why serving engines use continuous batching, where requests join and leave the batch between tokens, and manage the cache in pages. I'd also look at quantisation to fit the model on fewer GPUs, and I'd measure time to first token and time per output token, not just one latency number."

"I treat a prompt, a model version and retrieval settings like code: versioned, and every change goes through CI. The core is an eval set of a few hundred cases, drawn from real, anonymised traffic, plus every failure we've seen, each tagged by category and with what a good answer must contain or avoid. Scoring uses deterministic checks where I can: valid JSON, required fields, a citation present, no banned content. For open-ended quality I use a judge model with a clear rubric, and I check its grades against a sample of human labels, because a judge can be biased. The pipeline also records latency and token cost. The gate compares against the current version per category, so a gain in one area can't hide a regression in another. After release, flagged production answers get reviewed and added to the set."

"A few things are new. Cost scales with tokens, so I track input and output tokens per request and spend per feature, and alert when a prompt change or a runaway loop makes it jump. Latency splits into time to first token and total generation time. Quality can't be read off one number, so I sample live conversations and grade them, with a judge model checked against people, and I track signals like thumbs down, retries and refusals. For retrieval features I check whether answers are grounded in the documents fetched. Safety gets its own counters: guardrail blocks, injection attempts, any personal data showing up in output. And every call is logged with the prompt version, model version and retrieved context, within the privacy rules we follow, so when something goes wrong I can reproduce exactly what the model saw."