Deep Learning Interview Questions

"Each layer computes a weighted sum plus a bias, which is a linear function. If I stack two of those with nothing in between, I get W2 times W1 times x plus some bias, and W2 times W1 is just another matrix. So a hundred linear layers are mathematically the same as one, and the whole network is no better than linear regression or logistic regression. The activation function breaks that. Putting something like ReLU between layers means each layer can fold and bend the space, and stacking those bends lets the network approximate very complicated functions. The classic small example is XOR: no single straight line separates it, but one hidden layer with a non-linearity handles it easily. So depth only buys you anything because of the non-linear step between layers."

"Sigmoid squashes everything between zero and one. The problem is that for large positive or negative inputs it goes flat, so the gradient is close to zero, and even at its steepest the derivative is only a quarter. Multiply that through many layers and the early layers barely learn. Tanh is similar but centred on zero, which helps a little, yet it still saturates. ReLU is just max of zero and x. For positive inputs the gradient is exactly one, so it doesn't shrink the signal, and it's very cheap to compute. The weakness is dying ReLU: if a unit's input ends up negative for every example, its gradient is zero and it never recovers. Leaky ReLU or GELU soften that. I'd still use sigmoid at the output when I need a probability for a binary or multi-label decision."

"Softmax takes the raw scores from the last layer and turns them into a probability for each class. Cross-entropy then looks only at the probability the model gave the right class and takes the negative log of it. So if the model says 0.9 for the right answer the loss is small, and if it says 0.01 the loss is large. That's also exactly maximising the likelihood of the labels. The nice part is the gradient: with respect to the logits, it works out to predicted probabilities minus the one-hot label, so a confidently wrong prediction gets a big correction. With mean squared error on top of softmax, the gradient gets multiplied by the softmax slope, which is tiny when the output is saturated, so a badly wrong model can learn slowly. In code I pass logits straight into the built-in cross-entropy, which does the log-softmax in a stable way."

"If every weight starts at zero, or at any single shared value, then every unit in a layer sees the same inputs with the same weights, so they all produce the same output and receive the same gradient. They stay identical through training, and a layer of five hundred units behaves like one. Random initialisation breaks that symmetry. But the size of the random values matters too. Too large and activations grow layer after layer until things saturate or overflow. Too small and they shrink towards nothing, and so do the gradients. Xavier, or Glorot, initialisation sets the variance based on the number of inputs and outputs, which suits tanh and sigmoid. He initialisation uses a variance of two over the fan-in, which compensates for ReLU zeroing out half its inputs. Biases are usually fine starting at zero."

"Training needs to know, for every weight, how the loss would change if I nudged that weight a little. Backpropagation is the efficient way to get all of those at once. First I run the forward pass: inputs go through each layer, I get a prediction and a loss, and I keep the intermediate values around. Then I go backwards. At the output I know how the loss changes with the prediction. Using the chain rule, I pass that back to the last layer's weights and to its inputs, and the gradient with respect to its inputs becomes the starting point for the layer before. Because each layer reuses the gradient handed down to it, the whole backward pass costs about the same order as the forward pass. Then a separate step, the optimiser, uses those gradients to update the weights."

"I'll keep inputs as rows, so X is N by D. Forward: z1 is X times W1 plus b1, h is ReLU of z1, z2 is h times W2 plus b2, and p is sigmoid of z2. The loss is binary cross-entropy averaged over the batch, with a small epsilon inside the logs. Going back, the useful fact is that sigmoid plus cross-entropy gives a gradient at z2 of just p minus y, and I divide by N because of the mean. The gradient for W2 is h transposed times that, and for b2 it's the sum over the batch. To get back into the hidden layer I multiply by W2 transposed, then mask with z1 greater than zero, because ReLU only lets gradient through where it was active. Then W1 and b1 follow the same pattern with X. Before trusting it, I'd run a finite-difference check on a tiny network."

"During backprop the gradient at an early layer is a product of many terms, roughly one per layer between it and the loss. If those terms are mostly smaller than one, say sigmoid derivatives, which are at most a quarter, the product shrinks exponentially and the early layers stop learning. That's vanishing. If they're mostly bigger than one, the product blows up, updates become huge and the loss can jump to NaN. That's exploding, and it's especially common in recurrent nets, where the same weights are multiplied at every time step. For vanishing, I'd use ReLU or similar activations, He or Xavier initialisation, batch or layer normalisation, and residual connections that give the gradient a short path back. For exploding, gradient clipping by global norm is the standard fix, plus a lower learning rate. And I'd confirm the diagnosis by logging gradient norms layer by layer."

"Before residual connections, people saw that stacking more plain layers could make training error worse, not just validation error. That means it wasn't overfitting; the optimiser simply couldn't find a good solution, even though a deeper net could in principle copy a shallower one and set the extra layers to identity. A residual block changes the target. Instead of learning a full mapping, the block computes F of x and adds x back, so it only learns a correction. If a layer isn't useful, pushing F towards zero is easy. The other view is the gradient: the derivative of x plus F of x with respect to x includes an identity term, so gradient flows straight through the shortcut to earlier layers without being shrunk by every block on the way. When the block changes the number of channels or the resolution, the shortcut uses a one-by-one convolution to match shapes."

"First I'd make it reproducible. I'd find the step where it happened, reload the checkpoint just before, and save the exact batch. Then I'd check that batch: one corrupt input or an absurd target value can blow up the loss. If the data is clean, I'd look at the maths in anything custom: a log of something that can hit zero, a division by a variance or norm that can be zero, or a softmax I wrote myself. I'd swap those for the framework's stable versions or add a small epsilon. Next I'd look at gradient norms in the logs. If they spike just before the NaN, the gradients are exploding, so I'd lower the learning rate, check the warmup, and add clipping by global norm. If I'm training in 16-bit, I'd check for overflow and make sure loss scaling is on, or move to bfloat16. Anomaly detection modes can point at the first operation that made the NaN."

"Plain SGD takes a mini-batch, computes the gradient and steps against it, scaled by the learning rate. It's noisy and it zig-zags in narrow valleys. Momentum keeps a velocity, an exponentially decayed average of past gradients, and steps along that. Directions that stay consistent build up speed and directions that flip back and forth cancel out. Adam keeps that first moment and adds a second one: a running average of squared gradients per parameter. It divides the step by the square root of that, so parameters with large, noisy gradients take smaller steps and rarely updated ones take larger steps. It also corrects both averages early in training because they start at zero. Adam is usually my first choice because it converges quickly with little tuning. But on some image classification problems, SGD with momentum and a good schedule reaches equal or better final accuracy, so if I have the time to tune, I'll compare both."

"The learning rate is the first thing I tune because nothing else matters much if it's wrong. If it's too high the loss jumps around or shoots up to NaN; if it's too low the loss goes down so slowly I run out of budget. A quick way to find a sensible value is a range test: train for a few hundred steps while raising the learning rate exponentially, plot loss against rate, and pick something a bit below the point where the loss was dropping fastest. Warmup means starting with a small rate and ramping up over the first steps. Early on, the weights are random and Adam's running averages are unreliable, so a big step can knock training off course; warmup matters most for transformers and large batches. Decay does the opposite at the end: big steps early to explore, smaller steps later to settle. Cosine decay and step decay are common, and reduce-on-plateau works when I don't know the length in advance."

"With plain SGD, adding an L2 penalty to the loss puts an extra lambda times w into each gradient, and the update shrinks every weight a little each step. That's why people use L2 and weight decay as the same thing. In Adam they come apart. If I put L2 in the loss, that lambda times w term goes through Adam's adaptive scaling, so it gets divided by the square root of the running squared gradient. Weights that have had large gradients end up barely decayed, which isn't the regularisation I asked for. AdamW fixes this by decoupling: it computes the Adam step from the loss gradient alone, then shrinks the weights directly by the learning rate times the decay. Every weight decays at the same relative rate, and the decay setting becomes easier to tune separately from the learning rate. It usually generalises better, which is why it's the common default for transformers. I'd also exclude biases and normalisation weights from decay."

"Batch size touches both the hardware and the optimisation. On the hardware side, a GPU does best with lots of parallel work, so a bigger batch gives better throughput per example, up to the point where memory runs out. Memory grows with batch size mainly because every example's activations are kept for the backward pass. On the optimisation side, a small batch gives a noisy estimate of the gradient. That noise slows things down, but it can also help the model avoid sharp minima and generalise better. A big batch gives a smoother gradient and fewer steps per epoch, but if I don't raise the learning rate, progress per epoch drops, and very large batches can generalise worse without careful tuning. A common heuristic is to scale the learning rate roughly with the batch size and add warmup, then verify on validation. And with tiny batches, I'd avoid batch norm or switch to group norm."

"First I'd think about what's in memory: the weights, their gradients, the optimiser state, which for Adam is two extra tensors the size of the model, and the activations saved for the backward pass, which grow with batch size and input size. The cheapest fix is gradient accumulation: run smaller batches, call backward on each so gradients add up, and only step the optimiser every few batches. I get the same effective batch, just slower, though batch norm still sees the small batch. Next is mixed precision, running in 16-bit where it's safe, which roughly halves activation memory and usually speeds things up on modern GPUs. Activation checkpointing throws away some activations and recomputes them during the backward pass, trading extra compute for memory. If the model itself is too big, I'd look at sharding weights and optimiser state across GPUs, since plain data parallelism still keeps a full copy on each one."

"I'd want evidence before spending on a bigger model. First, where's the error? If training accuracy is also stuck well below target, the model is underfitting, and a bigger model or longer training could help. If training is near perfect and validation is behind, more capacity will probably make the gap worse, and data or regularisation is the better bet. Second, I'd train on a quarter, half and all of the data and plot validation accuracy. If it's still climbing at the full set, more labelled data is likely worth more than more parameters. Third, I'd read a few hundred mistakes. Often a chunk are wrong labels or cases even a person can't call, which means the ceiling is the data. I'd also raise serving cost, because a model several times larger may break the latency budget. Then I'd suggest one cheap experiment, like a moderately larger model on the current data, and let the result decide."

"During training, dropout randomly switches off each unit in a layer with some probability p, say 0.3, and it picks a fresh random set on every forward pass. That stops units from depending on particular other units being present, so the network learns more spread-out, robust features. You can think of it as training a huge family of thinner networks that share weights and averaging them at the end. At inference I want a deterministic answer, so dropout is turned off and every unit is used. To keep the expected size of the activations the same, frameworks use inverted dropout: during training they scale the surviving activations up by one over one minus p, so nothing needs rescaling at test time. The practical bug I watch for is forgetting to put the model in eval mode, which leaves dropout on and makes predictions noisy and worse."

"Batch norm takes each feature, or each channel in a conv net, and normalises it using the mean and variance of the current mini-batch, so it has roughly zero mean and unit variance. Then it applies two learnable parameters, a scale and a shift, so the network can undo the normalisation if that's better. In practice it lets me use higher learning rates, trains faster and makes the network less fussy about initialisation. The batch-to-batch noise also adds a small regularising effect. Why it works is still debated: the original reason given was reducing shifts in layer inputs, and later work pointed more at a smoother loss surface. At inference, there may be only one example, and I don't want a prediction to depend on what else is in the batch. So the layer keeps running averages of the mean and variance during training and uses those fixed values at test time."

"First I'd make sure the gap is real. I'd check that validation comes from the same distribution as production, that no near-duplicate images or users sit on both sides of the split, and I'd look at some validation mistakes to see if the labels themselves are wrong. If it's genuine overfitting, data is my first lever: more labelled examples if I can get them, otherwise stronger augmentation. Then I'd add early stopping on validation loss, which is nearly free. After that, weight decay and dropout, and if the model is clearly too big for the data, a smaller architecture or starting from a pretrained network and freezing most of it. I'd change one thing at a time and watch both training and validation curves, because piling on five regularisers at once tells me nothing about which one helped and can easily tip the model into underfitting."

"Data augmentation means making extra training examples by transforming the ones I have, so the model sees more variety without new labelling. For images that's random crops, flips, small rotations, colour and brightness changes, and blur. For audio it's adding background noise or shifting in time. The key question is whether the label is still true after the change. A horizontally flipped cat is still a cat, but a flipped road sign with a left arrow now means right, and rotating a handwritten six can turn it into a nine. So I choose augmentations that match variation the model will meet in real use, like different lighting or camera angles, and avoid ones that change meaning. I apply them only to training data, never to validation, and I always look at a grid of augmented samples before training, because a bug here quietly poisons everything."

"A 224 by 224 colour image has about 150,000 input values. A dense layer would give every hidden unit its own weight for every one of those, so even a modest layer runs into many millions of parameters, and it would ignore that pixels next to each other are related. A convolutional layer uses two ideas. Local connectivity: each filter only looks at a small patch, like three by three. And weight sharing: the same small filter slides across the whole image, so the layer has very few parameters and a pattern it learns, like a vertical edge, is detected wherever it shows up. Stack a few of these layers and the early ones pick up edges and colours, middle ones textures and shapes, and later ones whole object parts. That structure is a good fit for images, so conv nets need far less data than a dense net would."

"For each spatial side, the output size is the input size minus the kernel size, plus twice the padding, divided by the stride, plus one. Here that's 32 minus 5, plus zero, over 1, plus 1, which is 28. There are 16 filters, and each produces one output channel, so the output is 28 by 28 by 16. For parameters, the thing people miss is that a filter covers all the input channels, not just one. So each filter is 5 by 5 by 3, which is 75 weights, plus one bias, making 76. Times 16 filters gives 1,216 parameters. Notice the count doesn't depend on the image size at all; that's weight sharing. If I added padding of 2, the output would stay 32 by 32, and with stride 2 and no padding it would be 14 by 14, because I round down before adding one."

"A pooling layer slides a small window over each feature map and replaces the window with one number, usually the maximum or the average. The common setting is a 2 by 2 window with stride 2, which halves the height and width. It has no weights to learn. It cuts computation for the layers after it, and max pooling gives a bit of tolerance to small shifts, because the strongest response in a window survives even if it moves a pixel. The receptive field of a unit is the area of the original image that can influence it. A unit after one 3 by 3 convolution sees a 3 by 3 patch; after two stacked 3 by 3 convolutions it sees 5 by 5. Pooling and strides make it grow much faster, which is how deep layers end up seeing whole objects. Many newer designs use strided convolutions instead of pooling, and global average pooling before the classifier."

"A plain RNN reads one step at a time and updates a hidden state using the same weights at every step. When I train it, backprop through time pushes the gradient back through every step, multiplying by the same recurrent weights and activation derivatives each time. Over fifty or a hundred steps that product usually shrinks to almost nothing, or sometimes explodes, so the model can't learn that something early in the sequence matters later. An LSTM adds a cell state that runs alongside the hidden state and is updated mostly by addition. A forget gate decides how much of the old cell state to keep, an input gate decides how much new information to write, and an output gate decides what to expose as the hidden state. When the forget gate stays near one, information and gradient can pass along many steps without shrinking much. A GRU does something similar with two gates."

"Self-attention lets each position in a sequence build its new representation by looking at every other position and deciding which ones matter. Each input vector is multiplied by three learned matrices to make a query, a key and a value. The query is roughly what this position is looking for, the key is what each position offers, and the value is the information it will pass on. For one position, I take the dot product of its query with every key, which gives a relevance score for each position. Softmax turns those scores into weights that add to one, and the output is the weighted sum of the values. The scores are divided by the square root of the key dimension, because dot products grow with dimension, and very large scores push softmax into a near one-hot shape with tiny gradients. Multi-head attention runs several of these in parallel so different heads can track different relationships."

"Self-attention on its own has no idea about order. If I shuffle the inputs, each output is computed from the same set of keys and values, so the outputs just shuffle the same way. To fix that, position information is added. The original design added fixed sine and cosine patterns of different frequencies to each input embedding. Many models instead learn a position embedding, and others build relative position directly into the attention scores. On speed, an RNN has to finish step t before it can start step t plus one, so training can't be spread across the sequence. A transformer computes every position in one set of large matrix multiplications, which is exactly what GPUs are good at. It also helps learning: any two positions are connected in one attention step, so a dependency doesn't have to survive fifty recurrent steps. The price is that attention compares every pair of positions, so cost grows quadratically with sequence length."

"Two thousand images is too few to train a good image model from scratch, so I'd start from a backbone pretrained on a large general image dataset. I'd replace its final classification layer with a new one sized for my classes. First I'd freeze the backbone and train just the new head, so the random head doesn't send big, noisy gradients into good pretrained features. Once that's working, I'd unfreeze the top few blocks, or the whole network, and fine-tune with a learning rate maybe ten times smaller than before. I'd use the same resizing and normalisation the backbone was trained with, add augmentation, and keep a stratified validation set, since with small data the numbers are noisy. If my images look very different from the pretraining data, like medical scans, the early layers still help but I'd expect to fine-tune more of the network."

"I'd take a handful of raw inputs and run each one through the offline evaluation code and through the serving path, then compare the outputs. If they differ, I'd compare step by step: the preprocessed tensor first, then outputs layer by layer, until I find where they split. In my experience it's usually one of a few things. The model is still in training mode in serving, so dropout is randomly zeroing units and batch norm is using the statistics of a tiny serving batch. Or preprocessing differs: a different resize method, normalisation constants that don't match training, colour channels in the wrong order, or different text tokenisation. Or it's the artefact: an older checkpoint was deployed, some weights failed to load and were silently left at their initial values, or an export to lower precision changed the numbers. Once fixed, I'd add a test that runs the same fixed inputs through both paths on every release."

"At my last company I was training an image classifier for product categories, and validation accuracy sat at roughly chance for several epochs while training loss barely moved. Rather than tune hyperparameters, I went back to basics. The loss at the first step matched what I'd expect from random guessing, so the setup looked sane. Then I tried to overfit a single batch of thirty-two images, and even that failed, which told me the problem was the pipeline, not capacity. When I displayed images next to their labels, they didn't match. A recent change to the data loader sorted the file list but not the label list, so every image had a random label. Once I fixed the pairing, the model trained normally within an epoch. Since then, the first thing I do on any project is plot a grid of inputs with labels and make sure the model can overfit one batch."

"In one project we needed to spot surface defects on parts from factory photos, and we started with only about four hundred labelled images, most of them defect-free. Labelling needed an inspector's time, so it was expensive. I started from a pretrained image backbone, trained only a new head first and then fine-tuned the top blocks with a low learning rate. I used augmentations that matched real variation, like lighting changes, small rotations and blur, but avoided anything that could hide or invent a defect. Instead of labelling at random, I ran the model over unlabelled photos and sent the inspector the ones it was least sure about, which gave us far more useful examples per hour. Because the validation set was small, I used cross-validation and reported recall on defects, not overall accuracy. We reached the inspection team's target after two labelling rounds, and I learned that choosing what to label mattered as much as the model."

"On a churn prediction project at my last company, the team was keen to use a neural network on our customer table, and I built one with embeddings for the categorical fields. To be fair to it, I also trained a gradient-boosted tree model on the same splits with the same features. After tuning both, the tree model matched or beat the network on our main metric, trained in minutes instead of hours, and was much easier to explain to the retention team through feature importance. I shared both results side by side with the training cost and the serving latency and recommended the tree model. There was some disappointment, so I made the point that for mostly tabular data with a few dozen columns, that result is common, and that deep learning earned its place in our image and text work. We kept the network code in case the data grew."