Using an Azure Virtual Machine Scale set as Azure DevOps agents
Over the past few weeks, I’ve been implementing a few new networking features in Azure for a client. We did that to make our infrastructure more secure. I’ve been playing around with VNET’s, Private Links, Services Endpoints and Access Restrictions on Azure Web App, SQL databases, Storage and KeyVault. One of the issues I stumbled upon quite quickly was the fact that when you restrict access to a resource, Azure DevOps can’t reach that anymore as well. That means no deployments to a web app or running a migration on your SQL database. One of the things you could do is whitelist all the IP-addresses that the Microsoft-hosted agents could possibly use in Azure DevOps. The downside of that is that that list of IP-addresses is enormous and potentially changes every week.
After quite some research I found that the only real solution is to create your own Azure DevOps agents in Azure. You can then add those to a VNET that you can allow access to your resources. A recently added feature in Azure DevOps now allows you to use virtual machine scale sets to be used as an agent pool. That is pretty awesome because that still gives you the scalability of the hosted agents while minimizing costs. There are three steps we need to take here:
- Create a VM image to use as the base
- Create the VM Scale set
- Create the agent pool in Azure DevOps
Create a VM image to use as the base
Another benefit of using the Microsoft hosted agents in Azure DevOps, besides scalability, is that you don’t have to worry about installing software (sometimes you do when you have fancy requirements) and maintaining that. Luckily, the scripts that Microsoft uses to create those agents are open-sourced! That readme file describes how to create the VHD we need so just follow those steps. Come back here ones that readme continues with creating the Virtual Machine since we don’t need that. I’ve created a Ubuntu 18.04 image.
Now that we’ve created the vhd, we need to create an image out of that so we can, later on, use that to create the VM Scale Set. Use the following command to do that:
az image create -g <your-resource-group> -n <name> --os-type Linux --source <url-to-vhd-in-blob>
Create the VM Scale set
With the image in place, we can now create the VM Scale Set. To do that we just need one az command.
az vmss create \ --name releaseagents-1-Ubuntu1804 \ --resource-group release-agents \ --image ReleaseAgentImage-Ubuntu1804 \ --vm-sku Standard_D2_v3 \ --storage-sku StandardSSD_LRS \ --authentication-type SSH \ --instance-count 2 \ --disable-overprovision \ --upgrade-policy-mode manual \ --single-placement-group false \ --platform-fault-domain-count 1 \ --load-balancer “” \ --vnet-name vnet-release-agents \ --vnet-address-prefix 10.1.1.0/24 \ --subnet snet-release-agents \ --subnet-address-prefix 10.1.1.0/24
A few switched were added to the create command because of specific Azure DevOps requirements: –disable-overprovision, –upgrade-policy-mode manual, –load-balancer “”. I added –vnet-name and –subset since I want to add this scale-set to a VNET I already manage. If you are using these agents only to do releases, you might want to pick an even smaller VM size.
Create the agent pool in Azure DevOps
Added a new pool using the just created VM Scale set is perfectly described in the Microsoft docs