Chaining Bicep Deployments using Outputs and Stage Dependencies in Azure Pipelines

Sometimes, deploying a Bicep template using the preferred main.bicep method is not possible due to insufficient deployment permissions, especially when deploying across different subscriptions. This requires finding alternative ways to deploy your Bicep template, often involving context switching to obtain the necessary permissions.

In this blog, you will learn how to leverage Azure Bicep outputs and utilise them across different stages or jobs in your Azure Pipeline. This enables seamless chaining of Bicep deployments, each with its own deployment permission context. It’s almost the same way of setting dependencies in your main.bicep, but with the possibility of using different permission contexts.

The Problem

One issue you might encounter when using Azure Bicep is that your deployment principal may not have the necessary permissions for all the subscriptions (or maybe even on resource group level) involved in your deployments.

For example, from a platform team perspective, deployments might involve cross-subscription scenarios, such as connectivity, management, or identity subscriptions. Ideally, your deployment principal would have permissions at the platform-level management group, granting access to all these subscriptions. If that is the case, you can use a main.bicep file with multiple deployment scopes without any issues. However, this isn’t always possible because of least privilege implementations.

Let’s take a look at the following scenario:

In the scenario above, there are two Bicep modules, each with a different target scope (one targeting subscription A and the other targeting subscription B). These modules are defined in the main.bicep file using module declarations. The main.bicep file has a management group as its target scope and is deployed via an orchestration pipeline using a service connection with permissions on mg-Management. However, the service connection lacks deployment permissions for mg-Identity.

Before deploying the resources, the ARM engine verifies whether the necessary permissions are in place. In this case, there are insufficient permissions, and it throws an InvalidTemplateDeployment error with a message indicating that the deployment principal does not have permissions on Subscription B.

At the time of writing, Azure Bicep does not provide a “context switching” functionality. To address this, you need to create your own orchestration pipeline that utilises multiple contexts. In this blog, we demonstrate how to leverage Azure Bicep outputs and reuse them across stages, and using the readEnvironmentVariable function to retrieve these outputs. While this blog focuses on Azure Pipelines in Azure DevOps, the same approach can also be applied to GitHub workflows.

Let’s take a look at how to solve the problem.

The Solution

To solve the above problem, there are two possible approaches:

1. Grant the service connection additional permissions, allowing it to deploy to Subscription B. However, this may not be preferable in certain scenarios.
2. Modify the orchestration pipeline so that the Bicep templates are deployed using service connections with the correct permissions. This removes the main.bicep approach, but by leveraging Bicep outputs and configuring these as environment variables on the Azure DevOps agent we can somewhat maintain Bicep flexibility.

We are opting for option #2, as I prefer to limit the deployment principal’s permissions to what is strictly necessary in this context.

The orchestration pipeline

Below is the Azure Pipeline that orchestrates the deployment to Azure. It consists of two stages: the Management stage and the Identity stage. Each stage contains an AzurePowerShell task used to initiate the deployment and the task is configured to use a service connection:

• management-service-connection: with permissions on the management subscription
• identity-service-connection: with permissions on the identity subscription

Management stage

The goal of the Management stage is to deploy resources to the management subscription. The Bicep template file deploys a Log Analytics Workspace and outputs the resource ID of the Log Analytics Workspace as outLawResourceId. While this template only deploys a Log Analytics Workspace, additional resources can be added to the main.bicep file if needed.

The AzurePowerShell task includes the property name, which is used to reference the stage output variables. This property is important, because if it’s not set you are not able to use the refer to the output in the later stage.

In the Inline property, the deployment is orchestrated, and the output from the deployment is set on the pipeline using the task.setvariable command. In this example, the deployment output from the command New-AzResourceGroupDeployment is outLawResourceId and is assigned to the pipeline variable lawResourceId. Additionally, the variable is marked as an output variable using isoutput=true, allowing it to be reusable across stages and it to be accessible in the Identity stage.

Identity stage

The goal of the Identity stage is to deploy resources to the identity subscription. The Bicep template file in this stage deploys a Key Vault resource and configures the diagnostic settings to point to the Log Analytics Workspace deployed in the previous stage. For this to work, the lawResourceId from the Management stage must be retrieved.

To retrieve a pipeline variable from another stage, you must refer to the stage dependencies using the following syntax: $[stageDependencies.<StageName>.<JobName>.outputs['<TaskName>.<NameOfVariable>']].

When the StageName, JobName, TaskName, and NameOfTheVariable placeholders are filled in, it will look like this: $[stageDependencies.Management.MgmtInfrastructure.outputs['LoggingDeployment.lawResourceId']]

Additionally, the dependsOn property must be added because the Management stage needs to run first for the value to be set. For improved readability, create a variable for the Identity stage so you can avoid using the long stage dependency syntax in your pipeline.

The Identity pipeline now has access to the output from the Management stage, which can be used in the Azure Bicep template (Bicepparam) via the readEnvironmentVariable function:

param parWorkspaceId = readEnvironmentVariable('logAnalyticsWorkspaceResourceId')

If you want to learn more about combining Azure Pipelines, environment variables, and Azure Bicep, check out my post on how to work with environment variables in Azure Bicep.

Note: This blog illustrates how to use Bicep deployment outputs in Azure Pipelines and how to share them across stages. Suppose you want to implement this setup to configure diagnostic settings with the resource ID of the log analytics. In that case, you must also grant the Identity stage’s service connection permissions to read the keys of the Log Analytics Workspace. I created a custom role with the Microsoft.OperationalInsights/workspaces/sharedKeys/action permission to make it work.

End Result

After running the orchestration pipeline, it successfully deployed the Log Analytics Workspace, passed the Log Analytics resource ID value to the Identity stage, and configured the Key Vault to use the Log Analytics Workspace for logging purposes:

Conclusion

This is how you can deploy your Azure Bicep templates without compromising on specific access for your deployment principals, service connections, or service principals.

There are many ways to approach this, but this is my preferred method because it stays close to Azure Bicep’s flexibility by leveraging outputs and linking deployments together. Additionally, it does not require an Azure Pipeline task to retrieve values via Azure CLI or Azure PowerShell after the deployment is complete. This approach saves both deployment time and reduces complexity in the pipeline.

Note: While this setup is effective in the context of limited permissions, I recommend exploring how to achieve the same output directly with Bicep first, as this approach can become complex in larger projects. Keeping YAML configurations minimal is always a good practice in my opinion.