Machine Learning Interview Questions

"In supervised learning every training example comes with the answer, the label, and the model learns to predict that label for new inputs. If the label is a category, like spam or not spam, it's classification. If it's a number, like a house price, it's regression. In unsupervised learning there's no label at all. The model looks for structure on its own, like grouping customers with similar buying habits using k-means, or squeezing fifty correlated columns into a handful with PCA. The practical difference is data: supervised needs labels, which are often the expensive part, while unsupervised works on raw data but gives you groups or patterns you still have to interpret. There's also semi-supervised learning, where a few labels are combined with lots of unlabelled data."

"Bias is error from wrong assumptions, like fitting a straight line to a curved relationship. The model misses the pattern even on training data. Variance is error from being too sensitive to the particular sample you trained on, like a very deep tree that memorises noise. For squared error, the expected test error splits into bias squared, variance and irreducible noise. Making a model more flexible usually lowers bias but raises variance, so the goal is the sweet spot between them. To diagnose, I compare training and validation error. If both are high and close together, it's bias, so I add features or use a more flexible model. If training error is low but validation error is much higher, it's variance, so I get more data, add regularisation, or simplify. Learning curves make this very clear."

"A big gap like that means the model is overfitting: it has learned noise and quirks of the training set that don't generalise. Before fixing the model, I check the split is fair, because a validation set from a different period or population can create a gap on its own. Then I try the cheap fixes. I reduce complexity, for example capping tree depth or raising the minimum samples per leaf. I add regularisation, like L2 on a linear model. For boosting or anything trained in rounds, I use early stopping on the validation score. I also look at the features: dropping noisy or redundant ones often helps. And if I can get more data, that's the most reliable fix of all. I judge each change with cross-validation, not a single split."

"Both add a penalty on the size of the weights to the loss, and a strength parameter controls how hard we push. L1 adds the sum of the absolute values, L2 adds the sum of the squares. The difference shows in the gradient. With L2, the push toward zero is proportional to the weight, so as a weight gets small the push gets small too, and it settles near zero but not at it. With L1, the push is the same size no matter how small the weight is, so weak features get driven all the way to zero. Geometrically, the L1 constraint region has corners on the axes, and the best point often lands on one. So I use L1 when I want feature selection, and L2 when features are correlated, because L1 tends to keep one of a correlated group almost at random."

"Linear regression assumes the relationship is linear in the features, that the errors are independent of each other, that they have roughly constant variance across the range, and, for the classic tests and intervals, that they're roughly normal. It also struggles when features are highly correlated, because the coefficients get unstable even if predictions stay fine. In practice I care most about linearity and independence. If the relationship is curved, the model is just wrong, so I'd add transforms or interaction terms. If errors are correlated, like in time series, my error estimates are too optimistic. Normality mostly matters when I'm reporting p-values or intervals, not when I only need predictions. My main tool is a residual plot: a pattern or a funnel shape tells me which assumption is breaking."

"Logistic regression takes a linear combination of the features and passes it through a sigmoid, so the output is a probability between zero and one. Linear regression on a zero-one label can predict values below zero or above one, and a few extreme points can shift the line and move the decision boundary badly. For training, we use log loss, which is the negative log-likelihood of the labels. It punishes a confident wrong prediction very hard, which is what you want. And with the sigmoid, log loss gives a convex problem, so there are no bad local minima for gradient descent to get stuck in. Squared error combined with a sigmoid isn't convex, and its gradient almost vanishes when the model is confidently wrong. A nice side effect is interpretability: each coefficient is the change in log-odds for a one-unit change in that feature."

"An SVM looks for the boundary that separates the classes with the widest possible margin. Only the points closest to the boundary, the support vectors, decide where it goes, which is why it's called that. Real data overlaps, so we use a soft margin, and C controls how much we pay for points that break the margin. A high C tries hard to classify every training point, giving a narrower margin and more risk of overfitting. A low C accepts some mistakes for a wider, smoother boundary. The kernel trick lets the SVM draw curved boundaries. Instead of mapping the data into a higher-dimensional space, it uses a kernel function, like RBF, that gives the inner products in that space directly. SVMs need scaled features and get slow on very large datasets."

"A decision tree looks at every feature and a range of possible thresholds, and picks the split that makes the child nodes as pure as possible. For classification that's usually measured with Gini impurity or entropy, where entropy gives you information gain. For regression it's the reduction in variance, or squared error. It does this greedily, one node at a time, without looking ahead. Left alone, it keeps splitting until every leaf is pure, which means it memorises the training data. So I control it with a maximum depth, a minimum number of samples per leaf, or a minimum gain before splitting, or I grow it fully and prune it back using validation performance. A nice property is that trees don't need feature scaling, because a split only cares about order."

"A random forest trains many deep trees independently. Each one sees a bootstrap sample of the rows and a random subset of features at each split, so the trees are different from each other, and averaging them cancels out a lot of the variance. Adding more trees doesn't make it overfit, it just stops improving. Gradient boosting builds trees one after another. Each new, usually shallow, tree is fitted to the negative gradient of the loss, which for squared error is just the residuals, and a learning rate scales its contribution. That steadily reduces bias, but with too many rounds it will overfit, so I use early stopping. In practice I reach for a random forest as a strong baseline that needs little tuning, and boosting when I have time to tune and want the best accuracy on tabular data."

"K-means starts with k centres, usually chosen with k-means plus plus so they're spread out. Then it alternates two steps: assign every point to its nearest centre, then move each centre to the mean of the points assigned to it. It repeats until assignments stop changing. That minimises the total squared distance of points to their centres, but only to a local minimum, so I run it several times with different starts. To choose k, I plot that total distance against k and look for the elbow, check silhouette scores, and ask whether the groups are useful to the business. It fails on long, curved or very unequal clusters, it's pulled by outliers, and because it uses distance, features must be scaled first or the biggest-unit column dominates."

"PCA finds new axes for the data. The first is the direction along which the data varies most, the second is the most variable direction at right angles to the first, and so on. You keep the first few and drop the rest, choosing how many by the share of variance explained. Mechanically, you centre the data, usually standardise it so units don't dominate, and get the directions from the covariance matrix, in practice using SVD. I use it when I have many correlated features, to plot high-dimensional data in two dimensions, or to speed up something distance-based. I avoid it when people need to understand the features, because each component is a blend of everything. And since it ignores the label, it can throw away a low-variance direction that actually predicts the target."

"With one split, the score depends heavily on which rows happened to land in the test set, especially with small data. In k-fold, I split the data into k parts, say five. I train on four, validate on the fifth, and repeat so each part is the validation set once. Then I look at the mean score and also the spread, because a big spread tells me the model is unstable. For classification I use stratified folds so each fold keeps the class balance. If several rows belong to the same customer, I use group folds so a customer never appears on both sides. For time series I don't shuffle; I train on the past and validate on the next period. And I still keep a separate test set for the final number."

"Precision answers: when the model says positive, how often is it right? It's true positives over everything we predicted positive. Recall answers: of all the actual positives, how many did we find? It's true positives over all real positives. F1 is the harmonic mean of the two, so it's only high when both are high. For disease screening, missing a sick person is usually much worse than a false alarm, because a false alarm leads to a follow-up test while a miss can mean no treatment. So I'd push recall high and accept lower precision, and set the threshold with the doctors based on how many extra follow-ups they can handle. For something like a spam filter it flips, because putting a real email in spam is the costly mistake."

"ROC plots true positive rate against false positive rate. The problem with one fraud in a thousand is the false positive rate's denominator: all the legitimate transactions. I could flag thousands of good transactions and the false positive rate would still look tiny, so the ROC curve hugs the top left and AUC looks great. The precision-recall curve doesn't hide that, because precision compares false alarms to true frauds directly. A random model's PR AUC sits at the positive rate, here about one in a thousand, not at 0.5, so the baseline is much lower and gains are easier to judge. In the end I'd report precision and recall at the threshold the fraud team will actually run, like how many true frauds we catch if analysts can review a fixed number of alerts a day."

"MAE is the average absolute error, so if I'm predicting delivery time in minutes, an MAE of eight means we're off by eight minutes on average. It treats every minute the same and is less swayed by a few extreme cases. RMSE squares the errors first, so one miss of forty minutes hurts far more than four misses of ten. If big misses are what really cost us, RMSE is the better fit. Both are in the target's units, which makes them easy to explain. R-squared tells me how much of the variance I explain compared with just predicting the average. It's handy for context but has no units, and on a test set it can go negative if the model is worse than the mean. I usually report MAE or RMSE plus a naive baseline next to it."

"First I fix the evaluation, because accuracy is useless here: predicting 'negative' for everything scores almost perfectly. I use stratified splits so every fold has positives, and I look at precision, recall and the precision-recall curve. For training, my first move is class weights, so a mistake on a positive costs more in the loss. Resampling is the other option, undersampling the majority or oversampling the minority, possibly with SMOTE, but only ever on the training data, never the validation or test sets. Often the biggest win is simply moving the decision threshold away from 0.5 to match the real cost of a miss versus a false alarm. One catch: weighting and resampling shift the predicted probabilities, so if the business uses them as real probabilities, I recalibrate afterwards."

"Scaling matters whenever a model compares features by size. Distance-based models like k-nearest neighbours, k-means and SVMs will let a column in thousands swamp a column between zero and one. Gradient descent converges much faster when features are on similar scales. Regularised models need it so the penalty treats every weight fairly, and PCA needs it or it just picks the largest-unit column. Trees and tree ensembles don't care, because a split only depends on the order of the values. My default is standardisation, zero mean and unit variance. I use min-max when something needs a bounded range, and robust scaling based on the median when there are big outliers. The key rule is that I fit the scaler on training data only and reuse it everywhere else, ideally inside a pipeline."

"For a low-cardinality feature, like payment method with four values, I use one-hot encoding. Ordinal encoding is only right if the order means something, like small, medium, large, although tree models cope with arbitrary integer codes better than linear ones. For thousands of values, one-hot explodes the width and most columns are nearly empty. There I'd try frequency encoding, the hashing trick, or target encoding, where each category is replaced by the average target for that category. Target encoding is powerful but leaks badly if done naively, because each row's own label goes into its value. So I compute it out of fold and smooth rare categories toward the overall mean. I also plan for categories that appear only in production, mapping them to an unknown bucket."

"I add a column of ones so the intercept is just another weight, start the weights at zero, and loop. Each step I compute predictions, take the residuals, and the gradient of mean squared error is two over n times X transpose times the residuals. Then I step against the gradient, scaled by the learning rate. For the learning rate, I try a few values on a log scale, like 0.1, 0.01 and 0.001, and plot the loss. If it explodes or zigzags, it's too high; if it barely moves, it's too low. Scaling the features first helps a lot. Full-batch uses all rows per step, which is smooth but slow on big data. Mini-batch uses a small random chunk per step, which is noisier but far cheaper and is the usual choice in practice."

"I start by deciding which few hyperparameters actually matter for the model, like depth, learning rate and number of trees for boosting, and set sensible ranges, often on a log scale. Grid search tries every combination, which is fine for two small ranges but grows fast and wastes trials on settings that don't matter. Random search samples the space and, for the same budget, usually finds good values faster because it tries more distinct values of each parameter. When every fit is expensive, I use Bayesian optimisation, which builds a model of the score from past trials and picks promising next points. Whatever the method, I score with cross-validation or a validation set and use early stopping where I can. The test set stays untouched until the very end, or my final number is optimistic."

"Leakage is when training data carries information the model won't have when it's actually used, so offline scores look great and production doesn't. The classic kind is target leakage: predicting loan default with a column like 'sent to collections', which only exists because the customer defaulted. The other kind comes from the split: fitting a scaler or imputer on all the data before splitting, having the same customer in both train and test, or randomly splitting time-ordered data so the model trains on the future. To catch it, I'm suspicious of any score that seems too good. I look at feature importances, and if one feature dominates, I ask exactly when that value gets recorded. I also put all preprocessing in a pipeline so it's fitted inside each fold, and split by time or by group when the data needs it."

"I put the preprocessing and the model in one pipeline so nothing is fitted outside the training folds. A ColumnTransformer standardises the numeric columns and one-hot encodes the categorical ones, with unknown categories ignored so new values in the validation fold don't crash it. The pipeline chains that with logistic regression. Then I hand the whole pipeline to cross_val_score with stratified folds. On each fold, the scaler and encoder are fitted on the training part only and applied to the held-out part, exactly as they would be in production. I report the mean and standard deviation of the scores. The same pipeline object can go into a grid search, and at the end I fit it on all training data and save it as one artifact, so serving uses identical preprocessing."

"First I ask how fresh predictions need to be. If a daily score is enough, batch scoring into a table is simpler and cheaper than an online API. Then I package the preprocessing and model together as one versioned artifact, recording the training data snapshot and code version. The biggest risk is train-serve skew: a feature computed one way in a SQL training query and another way in the live service. So I make both paths use the same feature code or a shared feature store, log the exact features served with each prediction, and run a parity check comparing them against what the training pipeline would compute for the same entities. For rollout, I run it in shadow mode or on a small slice of traffic, compare against the current system, and keep a one-step rollback. Monitoring covers latency, errors, input distributions and, once labels arrive, accuracy."

"Data drift means the inputs have shifted, say a marketing campaign brings in younger users, while the underlying rules might be the same. Concept drift means the relationship itself changed, so the same inputs now lead to different outcomes, like fraudsters changing their pattern after we blocked the old one. Data drift I can watch immediately: I compare each feature's live distribution and the prediction distribution against the training data, using something like the population stability index or a KS test, and alert on big moves. Concept drift usually only shows once labels arrive, so I track the real metric on recent labelled data. When an alert fires, I don't retrain blindly. Often the cause is an upstream bug, like a unit change or a column suddenly full of nulls, and retraining on that would bake the bug in."

"At my last company I built a model to flag risky orders. Offline it separated good and bad orders really well, but in the first weeks live, it caught far fewer bad orders than we expected. I compared the features the live service had logged with what my training query produced for the same orders. One of the strongest features, the number of orders from that account in the last day, was computed from the full day of data in training, but in real time it only counted orders placed so far. So the model had learned from information it never had live. I rebuilt the feature so both paths used the same point-in-time logic, retrained, and results moved close to the offline numbers. Now I log served features from day one and run a parity check before any launch."

"In a churn project, I spent a week trying different models and the score barely moved. So I stopped tuning and read through examples the model got confidently wrong. Many customers labelled as churned came back a few weeks later. Our label said a customer had churned if they didn't log in for seven days, which caught people on holiday or with a quiet month. I took examples to the customer success team, and together we agreed on a definition based on thirty days of no activity plus no active subscription. With the new label the model improved more than any tuning had, and the scores made sense to the people using them. Since then, I always read a sample of labels and wrong predictions before touching hyperparameters."

"I built a model that ranked customers by how likely they were to cancel, for a retention team who'd never used a model before. They didn't care about AUC, so I didn't show it. I said, 'If you call the top two hundred customers on this list, about sixty of them would actually have left. If you picked two hundred at random, it would be around ten.' That made it concrete. I also showed three example customers and the main reasons the model flagged them, which built trust. And I was upfront that most people on the list wouldn't leave, so calls should feel like a check-in, not a rescue. They started with the top of the list each week, and I added a simple sheet showing how their saves compared with a random group."

"First, I'd tell my lead that evening, before anyone sees the slide. Presenting that number as a clean estimate would be misleading, and it's much worse if someone finds it later. Then I'd look for a fair number fast. Often there's newer data the model has never seen, like the most recent few weeks, and I'd score the model on it once. If there isn't, I'd re-run the tuning with cross-validation on the training data only and report that, clearly labelled. In the review I'd show the honest number and say plainly what happened, which usually earns more trust than it costs. Afterwards I'd set up the project so the test set lives in a separate place and is only read by a final evaluation script."

"I'd start by finding out what 'explain' has to mean. If there's a legal or policy requirement to give customers clear reasons, that's a hard constraint, and I'd check it with whoever owns compliance. Then I'd translate the small margin into real terms, like how many more good decisions per ten thousand applications. If that gain is small and explanations must be exact and consistent, I'd ship the logistic regression, where each factor's effect is visible. If the gain is meaningful, I'd look at options like adding monotonic constraints to the boosting model so it behaves sensibly, and using per-decision explanations such as SHAP values, then check with the business whether those explanations are good enough for customers. Either way it's a joint decision, and I'd write down why we chose it."

"First I'd make sure the gap is real and not noise from a small sample, by checking how many examples that group has and how stable the gap is across folds. Then I'd look for the cause. New customers, for example, often have little history, so the features the model leans on are mostly empty. That points to fixes like a simpler model or rules for that group, or adding features that don't depend on history. Then I'd take the numbers to the product owner with options: delay the launch to fix it, launch but route that group to the current system, or launch with close monitoring and a date for a fix. What I wouldn't do is launch quietly and hope nobody notices, especially if the weaker results could treat people unfairly."