I have always had a love-hate relationship with building Azure DevOps agent solutions. However, with Managed DevOps Pools, creating an agent solution is easier than ever. In this blog:
- You will learn about what Managed DevOps Pools is
- You will learn how Managed DevOps Pools is placed in Azure and what it does
- You will learn what kind of image types there are and how to set up private networking
- You will see Managed DevOps Pools in action
What is Managed DevOps Pools?
Managed DevOps Pools (MDP) is an Azure resource that integrates with Azure DevOps to create agent pools. These pools allow agents to be quickly spun up based on the configured demand. MDP is an evolution of the existing Azure Virtual Machine Scale Set (VMSS) resource. It essentially acts as a wrapper around VMSS, abstracting the management of infrastructure.
The underlying infrastructure, consisting of Virtual Machine Scale Sets (VMSS), for Managed DevOps Pools resides in a Microsoft subscription. This means that the critical infrastructure required to run Managed DevOps Pools is now Microsoft-managed (PaaS).
Managed DevOps Pools uses the hosted “on-behalf-of” model, which means that Microsoft hosts and manages all the infrastructure (virtual machines scale sets, networking, etc.) required to run Managed DevOps Pools behind the scenes. The scale sets in Microsoft’s environment are segmented and isolated using a dedicated virtual network.
A few benefits of Managed DevOps Pools:
- Less time spent managing and creating agent infrastructure.
- Easy Azure DevOps agent pool allocation, so each team can get their own pool of agents customisable with their own images.
- Scalable and reliable. Managed DevOps Pools can handle thousands of agents with reliable underlying infrastructure.
- Secure. Managed DevOps Pools is following the latest Microsoft security best practices.
High-level design
In the image below, you can see a high-level design of how Managed DevOps Pools works, what happens when you deploy a Managed DevOps Pools resource, and what the requirements are:
The high-level design above illustrates the deployment of a Managed DevOps Pools resource via an Azure Pipeline using the Microsoft-hosted agent. This represents the initial phase (greenfield) where no Managed DevOps Pool exists. After the Managed DevOps Pools resource is deployed, an agent pool is automatically registered in the specified Azure DevOps organisation, this is either at the organisation-level or project-level, depending on your configuration.
At this point, everything is set up to make use of the Azure DevOps agent pool and the underlying virtual machines (Virtual Machine Scale Sets) that Microsoft manages for you. As shown in the design, deploying a Managed DevOps Pool is very easy and hassle-free, thanks to automated management of the infrastructure.
In the “Managed DevOps Pools in Action” section of this blog post, you will see how to use the agent pool.
Requirements
To get started with Managed DevOps Pools there are some requirements you need to setup or need to take into account:
- Ensure that the Azure DevOps organisation is linked to the Azure Entra ID where the Managed DevOps Pools is deployed.
- The
Microsoft.DevOpsInfrastructureresource provider on the subscription must be registered. - When you do a deployment using a service connection from Azure DevOps, make sure that the underlying principal has the agent pools administrator permissions to register agent pools and that it has a stakeholder license to Azure DevOps. Service principals that need to do actions in Azure DevOps require a license.
- The resources Dev Center and Dev Center Project are required. More in the “Managed DevOps Pools in Action” section of this blog post.
Image types
There are several image types you can use. You can choose from pre-defined images created by Microsoft, or use more specialised images from the Azure Marketplace or through an Azure Compute Gallery.
| Image type | Description | OS Type |
|---|---|---|
| Microsoft Azure Pipeline Images | These images are equal to the Microsoft Hosted Agents and contain pre-installed software to deploy to and build on Azure. | Windows Server or Ubuntu |
| Azure Marketplace images | You can choose an existing image from the Azure Marketplace. | Windows Server, Windows, Ubuntu, Debian, RHEL or SUSE |
| Azure Compute Gallery Images | Managed DevOps Pools can be connected to an Azure Compute Gallery to load images from. These images can be specialised images used within and distributed by your organisation. | Windows Server, Windows, Ubuntu, Debian, RHEL or SUSE |
If you have used Microsoft Hosted Agents before and want to start with Managed DevOps Pools with the least amount of effort, then I would recommend starting with the Microsoft Azure Pipelines images because this offers a similar out-of-the-box experience.
Private Networking
By default, Managed DevOps Pools have their own virtual network segmentation, which is shared with Microsoft-hosted agents. If you want to use your own virtual network, you need to configure your Managed DevOps Pools to “join” it. To do this, you first need to setup a delegation on the subnet you created for Managed DevOps Pools and delegate it to the Microsoft.DevOpsInfrastructure/pools service.
After you have configured this delegation each virtual machine created for the Managed DevOps Pool will use the configured subnet. No other resources are then allowed to make use of the subnet.
At last, the service principal DevOpsInfrastructure requires permissions to manage the virtual network. This principal is already present in your tenant. Be aware that the principal object ID is different for each tenant. The following permissions are required:
- Reader & Network Contributor
- Or least privilege (custom role):
Microsoft.Network/virtualNetworks/subnets/join/actionMicrosoft.Network/virtualNetworks/subnets/serviceAssociationLinks/validate/actionMicrosoft.Network/virtualNetworks/subnets/serviceAssociationLinks/write
To retrieve the DevOpsInfrastructure ObjectID run the following Azure CLI command:
| az ad sp list --filter "displayname eq 'DevOpsInfrastructure'" --query "[0].id" |
Managed DevOps Pools in action
Let’s take a look at Managed DevOps Pools in action! In this section of the blog you will see:
- Infrastructure as Code Azure Bicep deployment of Managed DevOps Pools
- What is automatically configured in Azure DevOps
- How to use the agent pool in your Azure Pipeline
Infrastructure as Code (Azure Bicep) deployment
| resource resDevCenter 'Microsoft.DevCenter/devcenters@2024-02-01' = { | |
| name: 'my-dev-center' | |
| location: 'westeurope' | |
| } | |
| resource resDevCenterProject 'Microsoft.DevCenter/projects@2024-02-01' = { | |
| name: 'my-dev-center-project' | |
| location: 'westeurope' | |
| properties: { | |
| devCenterId: resDevCenter.id | |
| } | |
| } | |
| resource resManagedDevOpsPool 'Microsoft.DevOpsInfrastructure/pools@2024-04-04-preview' = { | |
| name: 'my-managed-devops-pool' | |
| location: 'westeurope' | |
| properties: { | |
| agentProfile: { | |
| kind: 'Stateless' | |
| } | |
| devCenterProjectResourceId: resDevCenterProject.id | |
| fabricProfile: { | |
| sku: { | |
| name: 'Standard_DS2_v2' | |
| } | |
| kind: 'Vmss' | |
| images: [ | |
| { | |
| wellKnownImageName: 'windows-2022/latest' | |
| } | |
| ] | |
| } | |
| maximumConcurrency: 1 | |
| organizationProfile: { | |
| kind: 'AzureDevOps' | |
| organizations: [ | |
| { | |
| url: 'https://dev.azure.com/john-lokerse' | |
| projects: [ | |
| 'DevOps-Playground' | |
| ] | |
| } | |
| ] | |
| } | |
| } | |
| } |
In the Bicep template above, a few resources are created:
- Microsoft.DevCenter/devcenters
- Microsoft.DevCenter/projects
- And finally, the Microsoft.DevOpsInfrastructure/pools
Managed DevOps Pools leverage the dev center resource, and within the dev center, a project is created. In short, a dev center allows teams to centrally manage projects, images, networking and compute regarding Dev Boxes, development environments and Managed DevOps Pools.
In this Bicep template, the Managed DevOps Pools resource is defined in its most basic form, without private connectivity, integration with the Azure Compute Gallery or any scaling configuration. It is configured to use the Microsoft Azure Pipeline Images via the wellKnownImageName property, using the windows-2022/latest value.
The agentProfile property defines whether the agents are Stateful or Stateless. In this template, the property is set to Stateless to refresh agents for each job. If the property were set to Stateful, you would be able to reuse the same agent for multiple builds.
It is recommended to use stateless pools because this improves the security of your deployments. However, if you need to retain build outputs (packages or files for example), you can use a stateful pool in combination with a configured lower recycle time (the maximum is 7 days) to balance security with agent reusability.
Finally, the organizationProfile is defined with the Azure DevOps information. In this case, my organisation is john-lokerse and the project I want to scope the to is DevOps-Playground. If the projects array is not set, then the pool will be deployed on organisation-level.
What is configured in Azure DevOps?
After the above template is deployed the Managed DevOps Pools is created in Azure, but also some Azure DevOps configuration is done. In the john-lokerse organisation under the DevOps-Playground project go to settings (1), click on Agent pools (2) and here you see the created agent pool (3):
The output of deploying the Managed DevOps Pools is an agent pool that can be used in Azure DevOps deployments. The name of the agent pool is the same as the name of the resource in Azure.
How to use the agent pool?
Now that the agent pool is registered, you can use it in Azure Pipelines. In the YAML file, assign the created agent pool, in this case, my-managed-devops-pool:
| trigger: | |
| - main | |
| pool: | |
| name: my-managed-devops-pool | |
| steps: | |
| - powershell: Write-Host "Hello Managed DevOps Pools!" | |
| displayName: '🚀 Run hello world!' |
When the pipeline is triggered, you will see the agent being assigned to the job and the agent being allocated:
In Azure, you can see the agent machine being spun up:
Conclusion
Managed DevOps Pools removes the burden of managing agents and is designed to simplify agent management by providing infrastructure under the hood, which you no longer need to manage. Making it essentially a Platform as a Service resource.
Additionally, the resource uses the same image as Microsoft-hosted agents, which is suitable for most scenarios, but the resource also offers the flexibility to use your own image. The best part is that the resource can integrate with your own network to access private endpoint-protected services.
All in all, Managed DevOps Pools saves a lot of time without compromising on security or flexibility!
Azure Cloud | John Lokerse 
hi how can i add a private gallery for Managed devpools ?
LikeLike
You need to have an Azure Compute Gallery with a generalized image. A generalized image has no machine specific information which means that it’s perfect for creating multiple virtual machines using the image.
After you have created the image and you have added it to your Azure Compute Gallery then you can add it to the Managed DevOps Pool: https://learn.microsoft.com/en-us/azure/devops/managed-devops-pools/configure-images?view=azure-devops&tabs=azure-portal&WT.mc_id=MVP_323261#azure-compute-gallery-images
LikeLike
and all this with terraform?
LikeLike
Works with Terraform too! There is no API differences between Bicep or Terraform, so the guidance written on the Microsoft Docs is the same. I don’t have Terraform expertise (currently) but you can get inspired by the Azure Verified Modules MDP module here: https://github.com/Azure/terraform-azurerm-avm-res-devopsinfrastructure-pool/blob/main/main.tf
It’s the same property that you have to refer to.
LikeLike