Kubernetes Interview Questions

"I think of it as a brain and a set of workers. On the control plane there's the API server, which is the only front door: kubectl, the nodes and every controller talk to it. Behind it is etcd, a key-value store holding the whole state of the cluster. The scheduler watches for pods that have no node yet and picks one for each. The controller manager runs loops that keep reality matching what we asked for, like keeping the right number of replicas. On a cloud there's often a cloud controller manager too, for things like load balancers. Each worker node runs the kubelet, which takes the pods assigned to that node and makes sure their containers are running, a container runtime like containerd that actually starts them, and usually kube-proxy, which sets up the rules so Service addresses reach the right pods."

"kubectl sends the manifest to the API server. It checks who I am and whether I'm allowed, runs admission controllers that can reject or adjust the object, and saves it to etcd. Nothing is running yet. The Deployment controller is watching, sees the new Deployment and creates a ReplicaSet. The ReplicaSet controller sees that and creates pod objects with no node. The scheduler notices unscheduled pods, filters out nodes that don't fit their requests, taints or affinity rules, scores the rest and binds each pod to a node. The kubelet on that node sees a pod assigned to it, asks the container runtime to pull the image, the network plugin gives the pod an IP, and the containers start. When the readiness probe passes, the pod's IP goes into the Service's endpoints and traffic starts flowing. Each step is a separate loop reacting to changes in the API server."

"etcd is the cluster's database. Every Deployment, Secret and pod status lives there, and only the API server reads and writes it. It's a distributed store that uses the Raft consensus algorithm, so you run an odd number of members, usually three or five, and it needs a majority to accept writes. With three members you can lose one. If it loses quorum, the API server can't make changes. Pods already running keep going, because the kubelet and the container runtime run them locally. But nothing new happens: I can't deploy, the autoscaler can't scale, and if a node dies its pods aren't replaced. So it's an outage of control rather than instantly of traffic, and it gets worse the longer it lasts. On a self-managed cluster I'd take regular etcd snapshots, keep them off the cluster, and practise a restore. On a managed service the provider looks after this."

"A pod is the smallest unit Kubernetes schedules. Usually it holds one container, but it can hold a few that need to live together, like an app plus a sidecar that ships its logs. Everything in a pod lands on the same node and shares one IP address and port space, so the containers talk to each other over localhost, and they can share volumes. Kubernetes schedules pods because some things genuinely belong together and have to be placed, started and stopped as one unit. The other thing I'd stress is that pods are disposable. If a pod dies or its node goes away, it isn't fixed in place; a controller like a Deployment creates a brand new pod with a new name and a new IP. That's why nothing should depend on a pod's IP, and why we put a Service in front of them."

"It's a chain of ownership. A pod on its own is one running instance; if it's deleted or its node dies, nothing brings it back. A ReplicaSet fixes that: it says I want three pods matching this label, and it creates or removes pods to keep that number. A Deployment sits on top and manages ReplicaSets. Every time I change the pod template, say a new image tag, the Deployment creates a new ReplicaSet and scales it up while scaling the old one down. The old ReplicaSet is kept around at zero replicas, and that's exactly what makes a rollback quick. So in practice I write Deployments and almost never touch ReplicaSets. If I scaled a Deployment's ReplicaSet by hand, the Deployment would simply set it back, because it owns that object."

"When I change the pod template, for example the image tag, the Deployment creates a new ReplicaSet and swaps pods over gradually. Two settings control the pace. maxSurge is how many pods above the desired count can exist during the update, and maxUnavailable is how many below it we're willing to go. It only moves on as new pods pass their readiness probe, so a good readiness check is what stops a broken version from taking over. I watch it with kubectl rollout status. If the release is bad, kubectl rollout undo goes back to the previous revision, or I can pick a specific one from rollout history. It's quick because the old ReplicaSet still exists and just scales back up. One catch: rollback only restores the pod template. If I also changed a ConfigMap or ran a database migration, those don't roll back with it."

"I start with apiVersion apps/v1, kind Deployment and a name. Under spec I set replicas to three. The part people trip on is the selector: matchLabels has to match the labels in the pod template, or the API server rejects it, because that's how the Deployment finds its own pods. The template is just a pod spec: a container with a name, an image and the port it listens on. I pin a real version tag rather than latest, so every rollout is tied to a specific build and a rollback actually goes somewhere. For production I'd add resource requests and limits and probes, but this is the minimum that runs. I'd apply it with kubectl apply -f, then check it with kubectl get deployment and kubectl rollout status."

"I use a Deployment when replicas are interchangeable: any pod can serve any request and it doesn't matter which one dies. A StatefulSet is for when each replica has an identity. It gives pods stable names, so I always get db-0, db-1 and db-2, and with a headless Service each one gets a stable DNS name the others can find. Each pod also gets its own PersistentVolumeClaim from a volumeClaimTemplate, and if db-1 is rescheduled it comes back attached to the same volume. By default it starts pods one at a time in order and removes them in reverse, which matters for clustered systems. So databases and message brokers fit. A web API that stores its data elsewhere is still a Deployment. And a StatefulSet doesn't replicate data for you; that's the application's job, it just gives each member a stable name and disk."

"It's a split between asking and providing. The pod references a PersistentVolumeClaim, which says: I need 20 gigabytes, ReadWriteOnce, from the fast storage class. A PersistentVolume is the actual storage, like a cloud disk or an NFS share. In most clusters nobody creates PVs by hand any more: the claim names a StorageClass, its provisioner creates a disk on demand and binds it to the claim, the disk is attached to the pod's node, and the kubelet mounts it into the pod. Two details matter in practice. Access mode: ReadWriteOnce means one node at a time can mount it, so two replicas on different nodes can't share it; ReadWriteMany needs storage that supports it, like a network file system. And reclaim policy: for dynamically provisioned volumes it defaults to Delete, so deleting the claim deletes the disk. For anything important I'd use Retain and keep backups."

"I wouldn't say no straight away, but I'd ask some hard questions first. Why not a managed database? If the reason is cost or portability, is that worth taking on backups, failover and upgrades ourselves? If we go ahead, I wouldn't hand-roll a StatefulSet; I'd want a mature operator that handles replication, failover and backups, and someone on the team who knows Postgres well. Then the platform side. How fast is the storage class, and what happens when a node dies? A ReadWriteOnce disk has to detach and reattach elsewhere, and a zonal disk can only move within its zone. During node upgrades, draining must never take down the primary and its replicas together, so we'd need disruption budgets and anti-affinity. And before any real data went in, I'd want a restore from backup and a failover drill actually done and timed."

"All three give a stable address in front of pods that come and go. ClusterIP is the default: a virtual IP and a DNS name that only work inside the cluster. Most services are like this, a backend that only other pods call. NodePort builds on that and also opens a port from a high range on every node, so anything that can reach a node's IP on that port reaches the Service. I'd use it for quick testing or when I run my own load balancer in front of the nodes, but it's rarely what I'd give users. LoadBalancer goes one step further: on a cloud it asks the provider to create a real load balancer with an external address. That's the simple way to expose one service publicly. For many HTTP services I'd rather use one Ingress than create a load balancer for each."

"The Service doesn't know pods by name. It has a label selector, say app equals web. A controller watches pods and keeps EndpointSlices up to date with the IP and port of every matching pod, marked ready or not. Only ready ones get traffic, so a pod that fails its readiness probe stops receiving requests without being restarted. Then on every node, kube-proxy watches Services and EndpointSlices and writes rules, usually iptables or IPVS, so a packet sent to the Service's virtual IP is sent on to one of the pod IPs. Some network plugins replace kube-proxy and do the same job with eBPF. On top of that, cluster DNS gives the Service a name like web.shop.svc.cluster.local, so pods in the same namespace can just call web. So when a Service answers nothing, the first thing I check is whether it has any ready endpoints."

"A LoadBalancer Service gives one external address per service and works at the connection level, so ten public services means ten load balancers. An Ingress works at the HTTP level. I can say requests for api.example.com go to the api Service and example.com/shop goes to the shop Service, all through one entry point. It also handles TLS in one place, using certificates stored as Secrets, often issued automatically by a tool like cert-manager. The catch is that an Ingress resource is only a set of rules. Nothing happens unless an ingress controller is running in the cluster, such as an NGINX-based one or a cloud provider's controller, and that controller is usually exposed through a single LoadBalancer Service itself. For newer setups I'd also look at the Gateway API, the newer and more expressive API for the same job."

"They work almost the same way: key-value data I can inject into a pod as environment variables or mount as files. ConfigMaps are for normal settings like a log level or feature flags. Secrets are for passwords, tokens and certificates. The important part is that a Secret's values are only base64 encoded, which anyone can decode. They sit in etcd without extra encryption unless encryption at rest has been turned on, which some managed providers do for you. What Secrets do give you is a separate object type, so RBAC can let a team read ConfigMaps but not Secrets. To make them properly safe I'd turn on encryption at rest, keep RBAC tight so very few people can read production Secrets, never commit real values to Git, and often pull them from an external secret manager with a tool that syncs them into the cluster."

"It depends on how the pod uses it. If the value comes in as an environment variable, it was read when the container started and will never change inside that container. If the ConfigMap is mounted as a volume, the kubelet does update the files, but only after a sync delay, and not at all if it was mounted with subPath. Even then the app has to notice and reload the file, and most don't. The quick fix is kubectl rollout restart on the Deployment, which replaces pods gradually so they pick up the new value. The better fix is to make config changes trigger a rollout by themselves. One common pattern is a checksum of the config in a pod template annotation, which many Helm charts do. Another is a new ConfigMap with a versioned name for each change, which also makes rollback cleaner."

"Helm is a package manager for Kubernetes. A chart is a folder of templated manifests plus a values.yaml file with defaults. Without it, I'd end up copying the same Deployment, Service and Ingress YAML for dev, staging and production and editing a few lines by hand, and the copies drift apart. With a chart the templates stay the same and each environment gets its own values file, with a different replica count, image tag or host name. When I run helm install or helm upgrade, Helm renders the templates, applies them and records the result as a release with a revision number, so helm rollback can take me back to an earlier revision. It's also how most third-party software is shipped, so installing something like an ingress controller is one command. Before an upgrade I like to run helm template or a diff to see exactly what will change."

"Readiness answers: should this pod get traffic right now? If it fails, the pod is taken out of the Service's endpoints but keeps running, so it can warm up or recover. Liveness answers: is this process stuck beyond saving? If it fails, the kubelet kills and restarts the container. Startup is for slow starters: until it succeeds, liveness and readiness don't run, so a service that takes two minutes to boot isn't killed halfway. Getting these wrong is expensive. The classic mistake is a liveness probe that checks the database. The database has a short blip, every pod fails liveness, every pod restarts at once, and a small problem becomes a full outage. So I keep liveness very simple, basically can the process answer at all, and put dependency checks in readiness if anywhere. I'd rather have no liveness probe than an aggressive one."

"Because start-up can take two minutes, I'd use a startup probe every five seconds with a failure threshold of 24. That allows up to 120 seconds before the kubelet gives up and restarts the container. Until it passes the other probes don't run, so I don't need a big initial delay on liveness. Readiness checks a separate ready endpoint every five seconds and takes the pod out of traffic after three failures, so roughly 15 seconds. Liveness checks a simple healthz endpoint every ten seconds and restarts after three failures, so the process has to be stuck for about half a minute before it's killed. I'd tune these from real data: how long starts actually take in production and how slow the endpoints get under load. The endpoints themselves should be cheap and shouldn't call other services."

"A request is what the container is guaranteed and what the scheduler uses to place it. If I request half a CPU and 512 megabytes of memory, the pod only goes on a node that has that much unreserved capacity. A limit is the ceiling at runtime, and the two resources behave differently when you hit it. CPU can be shared out over time, so a container over its CPU limit is throttled: it just runs slower. Memory can't be taken back, so a container that goes over its memory limit is killed, shows OOMKilled, and restarts. Requests and limits also set the pod's quality of service class. If every container has requests equal to limits for both CPU and memory, it's Guaranteed, and those pods are the last to be evicted when a node is short of memory. Pods with no requests or limits at all are BestEffort and go first."

"The HPA is a control loop. Every so often it reads a metric, most often average CPU utilisation from the metrics server, and compares it with the target. The rule is roughly: desired replicas equals current replicas times current value divided by target, rounded up. So four pods running at 90 percent against a 60 percent target becomes six. It stays between the min and max I set, and it scales down more cautiously than up, using a stabilisation window so it doesn't flap. The part people miss is that CPU utilisation is measured against the pod's request. If the container has no CPU request, the HPA can't calculate it and won't scale. Other reasons: the metrics server isn't installed, it's already at max replicas, or it did scale but the new pods are Pending because the nodes are full, which is a job for a cluster autoscaler."

"Namespaces split one cluster into named sections, usually by team or application, sometimes by environment. Names only have to be unique inside a namespace, and a lot of controls hang off them: Roles grant access per namespace, ResourceQuotas cap how much CPU and memory a team can claim, and LimitRanges set default requests. What they don't do on their own is isolate the network. A pod in one namespace can call a Service in another unless NetworkPolicies say no and the network plugin enforces them. They also don't isolate nodes: pods from different namespaces can share a machine and a kernel. And some objects aren't namespaced at all, like nodes, PersistentVolumes and ClusterRoles. So namespaces work well for separating teams that trust each other. For tenants that don't, I'd want separate node pools or separate clusters."

"RBAC has two halves: what is allowed, and who gets it. A Role is a list of rules, like get, list and watch on pods, in one namespace. A ClusterRole is the same but can cover cluster-wide things like nodes, or be reused across namespaces. A RoleBinding connects a role to subjects, meaning users, groups or ServiceAccounts, inside one namespace. A ClusterRoleBinding does it for the whole cluster, which is where I'm most careful. ServiceAccounts are the identities for workloads and pipelines; every pod runs as one. RBAC is purely additive, with no deny, so the only way to remove access is to remove a binding. In practice I write narrow Roles, often bind a built-in ClusterRole like view inside a single namespace with a RoleBinding, and always check the result with kubectl auth can-i, impersonating the account with --as."

"I'd agree the failures are a real problem, because nobody wants deploys blocked on permissions. But I'd push back on cluster-admin. CI systems are a favourite target, and if that credential leaks, whoever holds it can read every Secret and change anything in every namespace, production included. Instead I'd offer to fix it properly that same day. We look at exactly what the pipeline applies, usually Deployments, Services, ConfigMaps and maybe Ingresses, and create a ServiceAccount in each target namespace with a Role for just those resources and a RoleBinding. I'd test it with kubectl auth can-i before switching over. Where the platform allows, I'd use short-lived tokens or the CI system's workload identity instead of a long-lived token stored in CI. If something still fails later, the error names the missing permission, and adding one rule takes minutes."

"CrashLoopBackOff isn't the error itself. It means the container keeps exiting and the kubelet keeps restarting it, waiting longer each time. So I want the reason it exits. First, kubectl logs with --previous, because the current container may have only just started. Most of the time that shows it: a missing environment variable, a config file it can't parse, a database it can't reach at startup. If the logs are empty, I run kubectl describe pod and look at Last State. Exit code 137 with reason OOMKilled means it hit its memory limit. Exit code 1 is usually the app failing on its own. If the events show liveness probe failures, the kubelet is killing a container that's slow rather than broken. If the command or entrypoint is wrong, it dies instantly with no app logs, and describe shows the runtime error. Then I check what changed recently: image, config or Secrets."

"Pending with no node means the scheduler couldn't place it, and the good news is it tells you why. I run kubectl describe pod and read the events at the bottom. There'll be a FailedScheduling message, something like zero of five nodes available, three with insufficient memory, two with a taint the pod doesn't tolerate. The usual causes show up there. The pod's requests are bigger than the free allocatable capacity on any node, often because someone asked for far more than the app needs. A taint on the nodes, like a dedicated GPU pool, with no matching toleration. A nodeSelector or affinity rule that matches nothing, often a typo in a label. Or a PersistentVolumeClaim that isn't bound, or whose disk sits in a zone with no room. If the pod has a node but shows ContainerCreating or ImagePullBackOff, that's a different problem, on the node or registry side."

"I work from the Service towards the pod. First the endpoints: if the Service has no ready endpoints, either the selector doesn't match the pod labels, often a tiny typo, or the pods aren't ready. If there are endpoints, I check ports, because it's easy to point targetPort at 80 when the app listens on 8080. Next I port-forward to one pod. If that fails, the app itself is broken. If it works, I call the pod's IP from a throwaway pod. That catches a classic bug: the process listening only on 127.0.0.1. Port-forward still reaches it because it connects from inside the pod, but other pods can't. If the pod IP works, I call the Service name from a throwaway pod in the caller's namespace. If that works but the real caller still fails, I look at NetworkPolicies and at the exact name it's calling."

"My first move is to stop the damage, not to understand it. If the timing lines up with the rollout, I run kubectl rollout undo on that Deployment straight away. It's cheap and reversible; if the release turns out not to be the cause, we roll forward again later. While it rolls back I check whether the errors come only from new-version pods, using the version label in our metrics or logs, and I post in the incident channel what I did and what I'm watching. Once errors are back to normal, I look at why this got through. Usually the readiness probe said the new pods were fine while they were failing real requests, or the release depended on a config or database change that wasn't in place. Then I close the gap, for example with a better readiness check or a canary step that watches error rates before the rollout continues."

"At my last company our checkout API started returning errors every few minutes after a routine release. The pods said Running, but the restart count kept climbing. kubectl describe showed the containers were being OOMKilled. The new version had added an in-memory cache, and under real traffic memory crept past the 512 megabyte limit. It got worse because every restart emptied the cache, so the fresh pod was hammered with cache misses. I rolled back first to stop the damage, which took about two minutes. Then we load-tested the new version in staging, capped the cache size in the app, and set the memory request and limit from what we actually measured. Afterwards I added alerts on container restarts and on memory close to the limit, so next time we'd see it before customers did."

"In my last role I helped move a reporting service from two long-lived virtual machines onto our cluster. Building the image was the easy part; the real work was in the app. It read settings from a file on disk, so we moved them to environment variables from a ConfigMap, with passwords in Secrets. It wrote logs to a local file that would vanish with the pod, so we switched to standard output for the log collector. It saved generated reports on local disk, which breaks as soon as you have two replicas, so we moved them to object storage. We added a readiness endpoint and handled SIGTERM properly, so it finished in-flight requests before shutting down during rollouts. Afterwards deploys went from a manual evening job to a routine daytime rollout. What I'd do differently is measure real usage for requests earlier; our first guesses wasted a lot of capacity."

"On a previous team our search service had latency spikes, but CPU usage on the dashboards looked low. When I checked the container throttling metrics, it was being throttled heavily. It had a CPU limit of half a core, and although its average use was low, it burst hard for short moments on every request, and the limit clipped those bursts. At the same time its memory request was four times what it ever used, so nodes looked full while being half empty. I pulled two weeks of usage, including our busiest day, set the CPU request near the typical peak, removed the CPU limit for that service after agreeing it with the platform team, and cut the memory request to real usage plus headroom while keeping a memory limit. The latency spikes stopped, and the same pods fitted on noticeably fewer nodes."