This page is for engineers facing an Azure DevOps round, from a first build-and-release job to a senior platform role. Most interviews open with Boards and Repos, then spend the longest on YAML pipelines: stages, jobs, variables, templates and conditions. After that come environments and approvals, service connections, variable groups linked to Key Vault, artifacts and agents. Senior rounds add a broken deploy to talk through and a judgement call. Each question shows what the interviewer is checking, the shape of a strong answer and a short answer you can say out loud.
Search all questions by round, difficulty and level, or save the ones you want to practise.
Structure: an organization holds projects; each project holds the services.
Services: Boards for work, Repos for code, Pipelines for CI/CD, Test Plans for manual testing, Artifacts for packages.
Flow: a story on the board links to a branch, a pull request, a pipeline run and a deployment.
"An Azure DevOps organization holds one or more projects, and each project has five main services. Boards tracks the work: epics, stories, bugs and sprints. Repos hosts the Git repositories, with pull requests and branch policies. Pipelines builds, tests and deploys the code. Test Plans covers manual and exploratory testing, and Artifacts hosts package feeds like NuGet or npm. In a normal week, a developer picks a story from the sprint board, creates a branch from it, and opens a pull request that's linked back to the story. The pull request triggers a validation build, and once it's merged the pipeline builds once and deploys through dev, test and production. At the end, the story shows every commit and deployment it went into. A project can switch off services it doesn't use, for example if the code lives on GitHub."
Describing Azure DevOps as only a CI tool, or mixing it up with the Azure cloud services it deploys to.
Hierarchy: Epic, then Feature, then the story level, then Task; bugs sit at the story or task level.
Processes: Basic, Agile, Scrum and CMMI each name the levels differently.
Customising: you copy a system process into an inherited one to add fields, states or types.
"The backlog is a hierarchy. An Epic is a big goal that might take a few months, Features break it into things you can ship, the story level is work a team can finish in one sprint, and Tasks are the hours of work inside a story. The names come from the process the project uses. In the Agile process the story level is called a User Story; in Scrum it's a Product Backlog Item; in CMMI it's a Requirement; and the Basic process just has Epics, Issues and Tasks. So if another team says Product Backlog Item, they're probably on Scrum. You can't edit the built-in processes directly, so when a team needs an extra field or a new state like Ready for QA, you create an inherited process from one of them and change that."
Treating the work item names as fixed everywhere, or not knowing that built-in processes must be inherited before they can be changed.
Area path: the what or who, a tree of product areas used for ownership.
Iteration path: the when, a tree of releases and sprints with dates.
Teams: each team picks its area paths and sprints, and that decides its backlog and board.
"Area paths answer what part of the product this is and, through that, which team owns it. They form a tree, like Web, Web slash Checkout, Mobile. Iteration paths answer when, so they're a tree of releases and sprints, each with start and end dates. Every work item has one of each. Several teams can share a project because each team is set up with its own area paths and the sprints it follows. The checkout team's backlog only shows items under the checkout area path, so they don't see the mobile team's stories, but a manager above them can include sub-areas and see everything. If an item lands on the wrong board, it's almost always because its area path is wrong, not a bug in Boards."
Saying area and iteration paths are just labels, or confusing iteration paths with tags.
How: create the branch from the work item, mention the item ID with a hash, or link it on the pull request.
Enforce: a branch policy can require a linked work item before a merge.
Gain: each item shows its commits, pull requests, builds and releases.
"The easiest way is to create the branch straight from the work item, so the link exists from the start. In Azure Repos I can also mention the item's number with a hash in a commit message or in the pull request, and I can add links on the pull request itself. For code on GitHub, the Boards integration uses the AB prefix with the item number. On main we usually turn on the branch policy that checks for linked work items, so nothing merges without one. When the pull request completes, it can move the linked items to done. The gain is traceability. From a story you can see every commit and pull request, which pipeline run built it and where it was deployed, which is useful in audits and when you're asking which release a fix went out in."
Only knowing how to add links by hand afterwards, with no idea how to enforce or automate them.
Review: a minimum number of reviewers, votes reset on new pushes, required reviewers for sensitive paths.
Quality: build validation, status checks and resolved comments.
Hygiene: linked work items and a limited set of merge types.
"Once any required policy is set on main, changes have to come in through pull requests, so nobody pushes straight to it. I'd start with at least one or two reviewers, and I'd reset approvals when new changes are pushed, so a reviewer can't approve one version and see another one merged. I'd add required reviewers by path, for example the platform team on the pipeline folder. Then build validation, so the pull request must build and pass tests before it merges, plus any external status checks like a security scan. I'd require comments to be resolved so feedback isn't ignored, require a linked work item for traceability, and limit merge types, often to squash, so main history stays readable. Each policy protects against something specific: unreviewed code, broken builds, lost feedback or changes nobody can trace."
Listing policies without explaining the risk each one covers, or relying on reviewers alone with no build validation.
Merge result: the build runs on a temporary merge of the source branch into the target, not the source alone.
Expiry: a passed build can expire as soon as the target branch moves, some hours after it moves, or never.
Settings: required or optional, automatic or manual trigger, and path filters.
"The validation build doesn't test the feature branch on its own. Azure Repos creates a temporary merge commit of the source branch into the target, and the pipeline builds that, so it tests what main would look like after the merge. The source branch variable points at a pull request merge ref, not the feature branch. That's also why a pull request with merge conflicts can't be validated until the conflicts are fixed. The catch is time. If main moves on after my build passed, my green result was for an older main. The policy's build expiration handles that: it can expire immediately when main is updated, a set number of hours after main is updated, or never. For a busy main I'd use immediate or a few hours, so two pull requests that each pass alone can't combine into a broken main."
Saying the build just tests the feature branch, or not knowing that a passed build can go stale when main changes.
Problem: what the inconsistency was costing, in incidents or time.
Approach: build it with a pilot team, make the standard path the easy path.
Pushback and result: what teams objected to, what you changed, what improved.
"At my last company, each of six teams had its own pipeline style, and two production incidents came from a service that skipped tests on a hotfix branch. I proposed shared templates and common branch policies on main. I didn't write them alone; I built the first version with one team that was already feeling the pain, so it solved real problems. The pushback was mostly about speed and control: teams worried the template would be slow and they'd lose flexibility. So I added parameters for the things they genuinely needed to change, kept the template's build time equal to or faster than theirs, and versioned it so nobody was forced onto a change overnight. I also showed the numbers from the pilot: fewer failed deploys. Within a quarter, five teams had moved, and the last one moved when it needed a feature only the template had."
Forcing the standard on everyone overnight with no pilot, or describing it as purely a technical task with no people involved.
Classic: built in the web designer; build pipelines and separate release pipelines, stored in the service.
YAML: the pipeline is a file in the repo, reviewed and versioned with the code.
Why YAML: history, pull request review, branch-specific changes, templates and reuse.
"Classic pipelines are built by clicking in the web designer. There's a build pipeline, and a separate release pipeline with stages, approvals and gates, and the definition lives inside Azure DevOps, not in your repository. YAML pipelines are a file that sits next to the code. That means every change to the pipeline goes through a pull request, has history and can be reverted like any other code. A feature branch can change its own pipeline without touching everyone else's, and you can share logic through templates. YAML also covers deployment through multi-stage pipelines and environments, so you don't need a separate release pipeline. Classic still exists and many older projects use it, and an organization can switch off creating new classic pipelines. For anything new, I'd write YAML."
Saying classic pipelines can't do deployments, or not knowing where each type stores its definition.
Steps: scripts or tasks that run in order inside one job.
Jobs: a set of steps that runs on one agent; jobs in a stage can run in parallel.
Stages: major phases like build, test and deploy; each waits for the previous one by default.
"A step is the smallest unit: a script or a task. A job is a list of steps that all run on one agent, one after another, so they share the same working folder. A stage groups jobs into a phase like Build or Deploy. By default each stage waits for the one before it, but jobs inside a stage have no dependencies, so they can run in parallel if there are enough agents. The thing people trip over is files. Two jobs can land on two different machines, so a file built in one job isn't there in the next. You pass it on by publishing a pipeline artifact and downloading it, or with output variables for small values. So in this example, Compile and Lint can run side by side, and Test starts only when both succeed."
trigger:
branches:
include: [ main ]
pool:
vmImage: ubuntu-latest
stages:
- stage: Build
jobs:
- job: Compile
steps:
- script: ./build.sh
- job: Lint
steps:
- script: ./lint.sh
- stage: Test
jobs:
- job: Unit
steps:
- script: ./run-tests.sh
Assuming files carry over between jobs, or thinking every step runs on a fresh machine.
CI trigger: on pushes, filtered by branches and paths; batching for busy branches.
PR trigger: in YAML for GitHub and Bitbucket; for Azure Repos, through the build validation policy.
Others: schedules, completion of another pipeline, and manual runs.
"The common one is the CI trigger: a push to a branch runs the pipeline, and I filter it by branch and by path, so a change to docs doesn't kick off a full build. On busy branches, batching makes it wait and build the latest commit instead of one run per push. For pull requests, there's a pr section in YAML, but it only applies to GitHub and Bitbucket repositories. For Azure Repos Git, it's ignored, and you set up pull request builds through a build validation branch policy instead. I've seen people lose a day on that. Then there are scheduled triggers with cron syntax, which by default only run if something changed since the last good scheduled run, and pipeline resource triggers, so a deploy pipeline can start when a build pipeline finishes. Setting trigger to none turns off the push trigger, so with no schedule or resource trigger it only runs by hand."
Adding a pr block to YAML for an Azure Repos repository and expecting it to work.
Macro: round brackets, replaced at run time just before each task runs.
Template expression: double curly braces, replaced when the YAML is compiled, before any job starts.
Runtime expression: square brackets, evaluated at run time, used in conditions and variable definitions.
"There are three forms. The macro form, a dollar sign with round brackets, is the everyday one. It's replaced at run time, just before a task runs, and if the variable doesn't exist, the text is left exactly as written, which is why you sometimes see the raw name in a log. The template expression form uses double curly braces and is replaced when the pipeline is compiled, before any agent starts. It's what you use for template parameters and for changing the shape of the pipeline, like adding stages in a loop. Because it runs so early, it can't see anything set by a script during the run. The runtime expression form uses square brackets and is evaluated at run time, and it's the one you use to read output variables from other jobs or build a value inside a variables block."
variables:
env: staging
isMain: $[ eq(variables['Build.SourceBranch'], 'refs/heads/main') ]
steps:
- script: echo "Deploying to $(env)" # macro, at run time
- script: echo "Compiled for ${{ variables.env }}" # template, at compile time
- script: echo "Is this main? $(isMain)" # runtime expression, read as a macro
Treating the three forms as interchangeable, or expecting a compile-time expression to read a value set during the run.
Set it: a logging command with isOutput=true, in a step that has a name.
Same job: read it as stepName.variable.
Other job or stage: map it through dependencies or stageDependencies, and depend on the producing job or stage.
"A normal variable set by a script only lives inside its own job. To share it, I set it as an output variable with the setvariable logging command and isOutput set to true, and the step must have a name, because that name becomes part of the path. Inside the same job I read it as the step name, a dot, then the variable name. From another job in the same stage I use the dependencies object, and from another stage I use stageDependencies, giving the stage, the job, then outputs with the step and variable name. I map it into a variable with a runtime expression, and the consuming stage must depend on the producing one, or the value comes back empty. Deployment jobs have a slightly different path because of the lifecycle hooks, so I check that when I use them."
stages:
- stage: Build
jobs:
- job: Version
steps:
- bash: echo "##vso[task.setvariable variable=appVersion;isOutput=true]1.4.2"
name: setver
- stage: Deploy
dependsOn: Build
jobs:
- job: Release
variables:
appVersion: $[ stageDependencies.Build.Version.outputs['setver.appVersion'] ]
steps:
- bash: echo "Deploying version $(appVersion)"
Expecting a variable set in one job to appear in the next job automatically, or forgetting to name the step.
Includes: step, job, stage and variable templates pulled in where you need them.
Extends: the pipeline extends one template that controls the overall shape.
Sharing: keep templates in their own repo, reference it as a resource, pin to a tag or branch.
"There are two ways to use templates. Include templates are files of steps, jobs, stages or variables that you drop into a pipeline, with typed parameters so each team passes its own values. Extends templates flip that around: the whole pipeline extends one template, and the template decides the structure, so a platform team can say every pipeline gets a security scan stage. To share across repositories, I keep the templates in a dedicated repo and add it under resources, repositories, then reference a file with an at sign and the repo alias. I pin that reference to a tag or a release branch, not main, so a change to a template doesn't break fifty pipelines at once. Teams move to a new version when they're ready, and the template repo has its own pull requests and tests."
# templates/build.yml in the shared repo
parameters:
- name: configuration
type: string
default: Release
steps:
- script: dotnet build --configuration ${{ parameters.configuration }}
- script: dotnet test --configuration ${{ parameters.configuration }} --no-build
---
# azure-pipelines.yml in an app repo
resources:
repositories:
- repository: shared
type: git
name: Platform/pipeline-templates
ref: refs/tags/v2
steps:
- template: templates/build.yml@shared
parameters:
configuration: Debug
Copying the same YAML into every repo, or pointing every pipeline at the templates repo's main branch with no versioning.
Default: a stage or job runs only if the ones it depends on succeeded.
Custom condition: replaces that default, so you add succeeded() yourself.
dependsOn: controls order; an empty list lets a stage start straight away.
"By default, a stage runs only if the stages it depends on succeeded. To limit production to main, I add a condition that checks the source branch equals refs heads main. The trap is that a custom condition replaces the default one. If I write only the branch check, the stage can run even when the test stage before it failed, and I'd be deploying a broken build to production. So I always wrap it in and, with succeeded first. There are other functions too: always to run no matter what, which is handy for clean-up or publishing test results, failed to run only when something broke, like a notification job, and succeededOrFailed. The dependsOn keyword sets the order, and giving a stage an empty dependsOn list lets it start in parallel with the first stage."
- stage: DeployProd
dependsOn: Test
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
jobs:
- job: Deploy
steps:
- script: ./deploy.sh production
Writing only the branch check and not realising the stage can now run after a failed test stage.
Context: what existed, how many pipelines, why move.
Approach: inventory, start with one service, run old and new side by side.
Hard parts and result: approvals, gates and variables moved to environments and groups; what improved.
"At my last company we had around twenty classic build pipelines and a big release pipeline per service, all edited by hand, and nobody knew who'd changed what. I started with an inventory: every task, variable and approval. Then I moved one low-risk service first. The build part was easy. The hard parts were in release: pre-deployment approvals had to become checks on environments, stage-scoped variables became variable groups per environment, and a couple of gates had no direct match, so we used a REST API check instead. For two weeks the new YAML pipeline deployed to dev and test while the classic one still owned production, and we compared results. Once we trusted it, we switched production and disabled the classic release. After that I turned the common steps into shared templates, and the other services moved much faster."
Describing a big-bang switch of every pipeline at once, or not mentioning how approvals and secrets were carried over.
Environment: a named target like staging or production, with deployment history and its own checks.
Deployment job: targets an environment and runs a strategy: runOnce, rolling or canary.
Differences: lifecycle hooks, no automatic source checkout, history per environment.
"An environment is a named deployment target, like staging or production, that you create in Pipelines. Its value is that it keeps a history of every deployment to it, which commits and work items went out, and it's where approvals and other checks are attached. A deployment job is a special kind of job that says which environment it targets and which strategy to use. The simplest is runOnce; there's also rolling, for VMs, and canary. Strategies have lifecycle hooks such as preDeploy, deploy, routeTraffic and postRouteTraffic, plus on failure and on success, so rollback steps have a clear home. One difference that catches people: a deployment job doesn't check out the repository by default, because it's meant to deploy an artifact you've already built, not rebuild the source."
- stage: Production
jobs:
- deployment: DeployWeb
environment: production
pool:
vmImage: ubuntu-latest
strategy:
runOnce:
deploy:
steps:
- download: current
artifact: webapp
- script: ./deploy.sh $(Pipeline.Workspace)/webapp
Describing an environment as just a variable name, or not knowing that approvals are attached to it.
Where: on protected resources such as environments, service connections, agent pools, variable groups and repos.
Why: the resource owner sets them, so a pipeline author can't delete them in a pull request.
Types: approvals, branch control, business hours, required template, exclusive lock, calls to external APIs or alerts.
"In YAML pipelines, approvals and checks aren't in the pipeline file. They're set on the resource being protected: an environment, a service connection, an agent pool, a variable group, a secure file or a repository. That's deliberate. If the approval were in the YAML, anyone who can edit the pipeline could delete it in their branch. Because it lives on the resource, the owner of production controls it. When a stage is about to use that resource, Azure DevOps runs all its checks and the stage waits. Besides human approvals, there's branch control, so only runs from main can use production, business hours, a required template, an exclusive lock so two runs don't deploy at once, and checks that call a REST API or look at monitoring alerts. If the checks don't pass before their timeout, the stage doesn't run."
Saying approvals are a keyword in the YAML, or that anyone who can edit the pipeline can bypass them.
Template: a central extends template that injects the scan and deploy stages.
Enforce: a required template check on the production service connection, the key that opens production.
Harden: branch control, restricted pipeline permissions and a versioned, owned template repo.
"Asking teams nicely doesn't hold up, so I'd make it the only way to reach production. First, the platform team owns a central extends template that includes the scan stage and the deploy steps, with parameters for the things teams really need to change. Because it's extends, the team's pipeline fills in the gaps but can't remove the scan. Then I put a required template check on the production service connection, since that's what actually lets a run touch production. Any run that doesn't extend the approved template, from the approved repo and ref, simply can't use it. I'd add branch control so only main can deploy, and I'd switch off access for all pipelines on those resources, so each pipeline is approved one by one. Finally, the template repo gets its own reviewers and version tags, since it's now the most sensitive code we have."
Relying on a wiki page or a code review to make teams include the scan, with nothing on the resource enforcing it.
Build once: build, test and publish one artifact; later stages only deploy it.
Per environment: its own environment, variable group or vault, and service connection.
Production guard: approvals, branch control, exclusive lock, health check and a rollback path.
"The Build stage compiles, runs unit tests and publishes one pipeline artifact. Every later stage deploys that same artifact, so what we tested is exactly what reaches production. Dev deploys automatically on every merge to main. Test runs after that and includes integration or smoke tests. Production is a deployment job against a production environment. Each environment has its own variable group linked to its own Key Vault and its own service connection, scoped only to that environment's resource group, so a dev pipeline can't touch production even by mistake. On the production environment and service connection I'd add an approval from a group, branch control for main, and an exclusive lock. After deploying, a smoke test checks health, and if it fails, the on failure hook or a slot swap puts the last good version back. Deployment history on the environment shows what's live."
Rebuilding the code separately for each environment, or using one service connection with access to everything.
What: a saved, permissioned link to an external system such as Azure, a registry or a cluster.
Auth: workload identity federation, so the pipeline gets short-lived tokens and nothing needs rotating.
Least privilege: narrow scope and role, pipeline permissions, and checks on the connection.
"A service connection is how a pipeline gets access to something outside Azure DevOps, like an Azure subscription, a container registry or a Kubernetes cluster. Tasks refer to it by name, so the credential is never in the YAML. For Azure, I'd use workload identity federation. The pipeline gets a short-lived token at run time, so there's no client secret stored in Azure DevOps that can leak or quietly expire and break a deploy. The older way, a service principal with a secret, still works, but someone has to rotate it. Then least privilege: I scope the identity to one resource group, give it a role like Contributor there rather than Owner on the whole subscription, and have separate connections for dev and production. On the connection itself, I don't grant access to all pipelines, and I add approvals or branch control for production."
Using one subscription-wide Owner connection for every pipeline, or pasting credentials into pipeline variables instead.
Link: a variable group linked to the vault through a service connection, or the Key Vault task.
Access: that identity needs read access to secrets, and the pipeline must be allowed to use the group.
Use: map the secret into the script's env; it's masked in logs but not exported automatically.
"The usual way is a variable group in the Library that's linked to a Key Vault. I pick a service connection and the vault, then choose which secrets to expose, and they become variables with the same names. The identity behind the service connection needs permission to get and list secrets, through an access policy or the Key Vault Secrets User role if the vault uses Azure roles. Values are fetched when the run starts, so a rotated secret is picked up on the next run without editing anything. The pipeline also has to be authorised to use the group. In a script, secret variables aren't turned into environment variables automatically, so I map each one into env on that step. They're masked in the logs, but I still never echo them or write them to files."
variables:
- group: payments-prod-secrets # linked to Key Vault
steps:
- script: ./run-migrations.sh
env:
DB_PASSWORD: $(DbPassword)
Storing production secrets as plain pipeline variables, or expecting secret variables to appear in scripts on their own.
Masking: replaces the exact value in logs; an encoded or split value slips through.
Real risk: whoever can change the YAML on a branch that gets the secret can send it anywhere.
Controls: branch control on resources, no secrets for fork builds, narrow permissions and short-lived credentials.
"No. Masking only looks for the exact secret value in the log and replaces it with stars. If someone base64-encodes it, splits it, or sends it to a web address with curl, masking does nothing. So the real question is who can run code that receives the secret. Anyone who can push a branch and trigger a pipeline that uses the variable group can make it do anything with the value. The controls sit around that. I put branch control on the variable group, service connection and environment, so only runs from main, which needs a reviewed pull request, can use production secrets. Builds of pull requests from forks don't get secrets by default, and I'd keep it that way. I'd limit job access tokens to the current project, prefer short-lived federated credentials, and review changes to pipeline files carefully."
Treating log masking as a security boundary, or saying a secret is safe because nobody can see it in the UI.
Found: what was wrong and how you discovered it.
Change: Key Vault, linked variable groups, federated service connections, rotation.
Safety and result: staged rollout, rotation order, what stopped it happening again.
"In one project, I noticed a database password stored as a plain pipeline variable in several release pipelines, and a service connection using a secret nobody knew the expiry of. I listed every place secrets lived, then moved them into Key Vault, one vault per environment, and linked each to a variable group so pipelines pulled values at run time. I switched the Azure service connections to workload identity federation, so there was no secret left to expire. The order mattered: I pointed pipelines at the vault first, confirmed dev and test deployed fine, then rotated the real passwords, because the old values had been visible to anyone who could edit those pipelines. To keep it fixed, we added branch control on the production variable groups and a pull request checklist item for pipeline changes. We haven't had a secret-related deploy failure since."
Moving secrets into Key Vault but never rotating the ones that had already been exposed.
Pipeline artifact: files from one run, kept with that run and passed to later stages or pipelines.
Feed package: a versioned package, such as NuGet, npm, Maven or Python, that others install.
Rule: deployable output of this app goes as a pipeline artifact; shared libraries go to a feed.
"A pipeline artifact is the output of one run: the zipped web app, a container manifest, a test report. I publish it in the build stage and download it in the deploy stages, or in another pipeline, and it's kept for as long as the run is retained. An Azure Artifacts feed is a package repository. It holds versioned packages like NuGet, npm, Maven or Python, plus Universal Packages for arbitrary files, and other projects install them with their normal package manager. So the rule I follow is: if it's the thing this app deploys, it's a pipeline artifact. If it's a library that other teams consume, it gets a proper version and goes into a feed. Older pipelines use build artifacts with a different task, but pipeline artifacts are the newer, faster option."
Pushing every build's app zip into a package feed, or not knowing that pipeline artifacts follow the run's retention.
Upstream sources: the feed proxies public registries and keeps a copy of each package used.
Views: local, prerelease and release views let you promote a version once it's proven.
Access: readers for most people, publish rights for pipelines, and blocking public versions of internal names.
"I'd create one feed and point every project's package config at it, instead of at the public registries. The feed has upstream sources for the public registries we use, like npm or NuGet, so when someone installs a public package through it, the feed saves a copy. If that version later disappears from the public registry, our builds keep working. Internal libraries get published to the same feed by their pipelines. Then I'd use views: new versions land in the local view, and once a version passes testing we promote it to prerelease or release, and consumers point at the release view. For access, most people only read; only the publishing pipelines can push. I'd also make sure a public package can't take over one of our internal names. By default, once we publish a name ourselves, the feed won't pull in public versions of that name unless a feed owner allows external versions for that package."
Letting every project pull straight from public registries with no feed, or giving everyone publish rights to the shared feed.
Hosted: a fresh, maintained VM for each job, thrown away after; nothing to patch.
Self-hosted: private network access, custom tools, bigger machines, warm caches.
Cost: you patch, secure, scale and clean the machines yourself.
"Microsoft-hosted agents are the easy default. Every job gets a fresh VM from a maintained image with common tools already installed, and it's thrown away when the job ends, so there's nothing to patch and no leftovers between builds. The downside is that nothing is cached between runs, you can't install special hardware, and the machine can't reach resources on your private network. I'd choose self-hosted when the pipeline has to deploy into a private network, needs licensed or unusual tools, needs a bigger machine, or when warm caches would save a lot of build time. The extra work is real. We have to patch the OS and the agent, secure the machine because it runs whatever the pipeline says, clean workspaces, and add capacity when the queue grows. Each agent runs one job at a time, so scale matters."
Choosing self-hosted agents by default without mentioning patching and security, or thinking hosted agents keep state between jobs.
Elastic: scale set agent pools or container agents that grow with the queue and shrink when idle.
Clean: fresh or reimaged agents per job, so no state leaks between runs.
Isolated: separate pools for untrusted builds and production deploys, least privilege on each.
"I'd stop treating agents as long-lived pets. One option is an Azure virtual machine scale set agent pool: Azure DevOps adds VMs when jobs queue up and removes them when they're idle, and it can tear a machine down after each job, so every build starts on a fresh one. Another is running agents as containers on Kubernetes with an autoscaler that watches the queue. There's also Managed DevOps Pools, where Microsoft runs the machines and you choose the image, size, scaling and network. Whichever I pick, I'd bake tools into the image instead of installing them during builds. For safety, I'd split pools: one for pull request builds that run untrusted code, and a separate, locked-down pool with network access to production that only deployment stages can use, protected by checks on the pool. The agent service runs as a low-privilege account, never as an admin."
Running pull request builds and production deployments on the same long-lived agent that has admin rights.
Stabilise: get users back on the last good version before you debug.
Compare: diff the failed run against the last good one: pipeline and template changes, agent image, variables.
Suspects: expired or changed credentials, permissions, agent changes, config or secret edits.
"First I'd get users back to a working state. Since the deploy stopped halfway, production is probably a mix of old and new, so I'd redeploy the last good artifact to production, or swap back a deployment slot if we use them, and tell the team what's happening. Then I'd debug. The code didn't change, but lots of things around it can. I'd read the failing step's log, rerun with system diagnostics on if I need more detail, and compare with the last successful run. Did the YAML or a shared template change? The Initialize job log shows the agent and image version, so did a hosted image update change a tool? Did a service connection's credential expire or lose a role? Did someone edit a variable group or rotate a Key Vault secret? Once I find it, I fix it and add a check so it can't fail silently again."
Rerunning the pipeline again and again without reading the log, or debugging for an hour while users are still seeing errors.
Fast path: a small pull request with one quick reviewer is usually just as fast.
If bypass is needed: only people with the bypass permission, with a written reason that's audited.
Afterwards: review the change after the fact and fix whatever made bypass feel necessary.
"I'd agree the fix is urgent, but I wouldn't remove the branch policies, because people forget to turn them back on. For a one-line fix, a pull request with one reviewer on a call is usually just as fast, and the validation build tells us the fix at least compiles before it goes to production. If the build is too slow to wait for, Azure Repos lets someone with the bypass permission complete the pull request while overriding the policies, and it records who did it and the reason they gave. I'd use that, not a direct push, and not a policy change. After the incident, I'd make sure someone reviews the change properly, and I'd look at why bypass felt necessary. Often the answer is a slow build or no clear hotfix process, and those are things we can fix."
Deleting the branch policy to push the fix and forgetting to turn it back on, or refusing to help at all during an outage.
Now: go to the environment's owner or administrator, not around the check.
Legitimate change: an admin adds a backup approver or bypasses the check once, and both are recorded.
Afterwards: approvals by a group with a minimum count, not one named person.
"First, I wouldn't try to get around it, like editing the YAML to point at a different environment or copying the deploy into a new pipeline. That would defeat the control, and the checks live on the resource anyway. I'd contact whoever administers the production environment and explain the urgency. They can add a backup approver, or, as the resource admin, bypass the check for this one run, and either way it's recorded who did it. If our process has an emergency change route, I'd use that. Once the deploy is out, I'd fix the real problem, which is a single named person. I'd set the approval to a group, like the release managers, with a minimum of one or two approvers, and make sure the group always has someone available. I'd also check whether people can approve their own runs, and turn that off for production."
Rewriting the pipeline so it skips the protected environment, or waiting silently until the approver returns.
ClapAssist is an AI interview assistant for Mac and Windows. It listens to the interview on your computer and shows you what to say, in short lines you can read while you talk. Your resume and notes are never stored on our servers. It stays out of screen share on every plan; only you can see it.