Validating YAML using Azure DevOps or CLI

Working with YAML can be a struggle. It does not matter how easy or complex the task is, an error is easily made. Errors might be an illegal character in a job name or incorrect formatting.

I work with YAML frequently to create or maintain CI/CD pipelines and for this post, I want to share how I validate my YAML files before these are added to the main branch. The validation can be done through the Azure DevOps web interface or via the CLI. The CLI can be a useful tool to automate this process in build validation.

Validation using the GUI

This is the YAML input we are going to work with:

This input contains an error in the job name. It contains an illegal character, which is the “space” character. Job names only allow [A-Z, a-z, 0-9, and underscores] characters.

Azure DevOps has a built-in validator that checks your YAML file for errors and invalidations. This validator also works with YAML templates.

To validate a pipeline follow these steps:

1. Go to the pipeline you want to validate.
2. Press on the Edit button on the top right.
3. On the edit-page click on the three dots on the top right.
4. (Optional) When using trunk based development you can choose the feature branch you are working in.
5. In this menu, click on validate and a message will appear with either OK if the pipeline is validated correctly or an error message if the pipeline is invalid.

The example input we are working with is invalid so after doing step 5 we get an error message containing the location of the error:

If we replace the spaces and the dash (-) with underscores (_) in the job name the pipeline will throw a success message:

Validation using the CLI

There is a way to validate through the CLI. This is useful because it is the perfect way to automate these validations. The YAML must be validated against an existing pipeline. Microsoft provides a “hidden and undocumented” API to validate YAML via a REST call:

https://dev.azure.com/$OrganizationName/$ProjectName/_apis/pipelines/$PipelineId/runs?api-version=5.1-preview

This REST call requires some given segments to find your Azure Pipeline:

• OrganizationName (e.g. Contoso)
• ProjectName (e.g. ContosoTeamProject)
• PipelineId, also known as BuildDefinitionId (e.g. 150)

The BuildDefinitionId can be found in the URL of the pipeline: https://dev.azure.com/contoso/ContosoTeamProject/_build?definitionId=150

This REST call is a POST, so it also requires a body:

$Body = @{
  "PreviewRun"   = "true"
  "YamlOverride" = <INSERT_YAML>
}

The success output is a bit vague because there is no OK statement. The output of the error message is clear and similar to the message shown in the GUI.

When the output is validated correctly this will be the output:

The full script with the YAML input provided from the last section:

Conclusion

This is how you can validate your YAML files using Azure DevOps and the CLI. This feature is useful when you are working with large YAML files or when you are using templated YAML files.

Update 10/06/2022:

I have written a PowerShell script that makes it easier to validate your YAML file. Also, I added an Azure Pipeline you can use as build validation. Check it out at my GitHub repo: https://github.com/johnlokerse/ado-yaml-validation