Move to Cloud

Use multi-stage DevOps pipeline to deploy Bicep Azure Landing Zone

Introduction

I have written about Bicep templates for Azure Landing Zone before. However if you have a look at the starter Azure DevOps pipeline you will notice that the complete deployment flow is covered by a single job with a bunch of tasks. This means that on every release you will run all modules (at once).

This is time-consuming, but also requires the infrastructure team(s) to be aware of common development practices like using feature branches to prevent accidental incomplete work to be deployed. In larger organizations you also have multiple teams working on different aspects of the Landing Zone setup.

One option could be to start spreading out into multiple pipelines (or even repositories), but personally I prefer to keep related things together. In the past I have deployed ALZ through (classic) release pipelines, which allow you to easily create multiple stages. However, YAML pipelines are taking over so it is time to see if we can do the same there.

Multi-stage YAML pipelines

As always, Microsoft has great documentation on multi-stage pipelines so be sure to read that as well.

In short, you can map a YAML pipeline to:

  • stage: Stages are a collection of related jobs. By default, stages run sequentially. Each stage starts only after the preceding stage is complete unless otherwise specified via the dependsOn property.
  • job: The jobs list specifies the jobs that make up the work of a stage. It is a collection of steps run by an agent or on a server.
  • step: Steps are a linear sequence of operations that make up a job.
  • Operation: An operation can take many shapes, some of them are task, checkout, powershell, bash, …

A job could also be a deployment job, which runs against a specific environment and has a deployment strategy. More on this in the next post.

The current pipeline

If we look at the sample starter Azure DevOps pipeline again, we can quickly see that these levels can be optional and the YAML script started directly at the jobs collection. Since there is only one job, it could even have started with the steps collection.

jobs:
- job:
  steps:

  - task: AzureCLI@2
    displayName: Az CLI Deploy Management Groups
    name: create_mgs
    inputs:

Defining a stage

In the classic release pipeline, you would have created a new visual block for your stage, here we’ll have to edit the YAML definition. It is very important to know that YAML is space-sensitive and each new collection increases the indentation by 2 spaces.

stages:
- stage: ManagementGroups
  displayName: Deploy Management Groups
  jobs:
  - job: DeplopyManagementGroups
    displayName: Deploy Management Groups
    steps:
    - task: AzureCLI@2
      displayName: Az CLI Deploy Management Groups
      name: create_mgs
      inputs:

I use Visual Studio Code, which correctly maps a tab to two spaces in YAML making life a lot easier. As a good practice, I also try to give my stages and jobs a name and display name.

The final result

It is up to you to define which tasks and/or jobs you want to combine in a single stage, I went for this combination.

Multiple Stages

Most of these stages still have multiple tasks which are related or can be combined, e.g. creating the networking resource group and creating the actual resources. Important to notice is that I have defined links between stages with dependsOn (line 43) to make sure some stages are deployed before others when executed completely.

It is still possible to select only a few stages, but then you have to check if the predecessors ran yourself.

Choose specific stages

And finally (a fragment) of the modified YAML pipeline:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
trigger: none

pool:
  vmImage: ubuntu-latest

variables:
  ServiceConnectionName: "ALZPipelineConnection"
  ManagementGroupPrefix: "alz"
  TopLevelManagementGroupDisplayName: "Azure Landing Zones"
  Location: "eastus"
  LoggingSubId: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
  LoggingResourceGroupName: "alz-logging"
  HubNetworkSubId: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
  HubNetworkResourceGroupName: "Hub_Networking_POC"
  RoleAssignmentManagementGroupId: "alz-platform"
  SpokeNetworkSubId: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
  SpokeNetworkResourceGroupName: "Spoke_Networking_POC"
  RunNumber: $(Build.BuildNumber)

stages:
- stage: ManagementGroups
  displayName: Deploy Management Groups
  jobs:
  - job: DeplopyManagementGroups
    displayName: Deploy Management Groups
    steps:
    - task: AzureCLI@2
      displayName: Az CLI Deploy Management Groups
      name: create_mgs
      inputs:
        azureSubscription: $(ServiceConnectionName)
        scriptType: 'bash'
        scriptLocation: 'inlineScript'
        inlineScript: |
          az deployment tenant what-if \
          --template-file infra-as-code/bicep/modules/managementGroups/managementGroups.bicep \
          --parameters parTopLevelManagementGroupPrefix=$(ManagementGroupPrefix) parTopLevelManagementGroupDisplayName="$(TopLevelManagementGroupDisplayName)" \
          --location $(Location) \
          --name create_mgs-$(RunNumber)          

- stage: DefinePoliciesRbac
  displayName: Define Custom Policies and 
  dependsOn: ManagementGroups
  jobs:
  - job:
    steps:
    - task: AzureCLI@2
      displayName: Az CLI Deploy Custom Policy Definitions
      ..
    - task: AzureCLI@2
      displayName: Az CLI Deploy Custom Role Definitions
      ..

- stage: ApplyPoliciesRbac
  displayName: Apply Custom Policies and 
  dependsOn: DefinePoliciesRbac
  jobs:
  - job:
    steps:
    - task: AzureCLI@2
      displayName: Az CLI Deploy Role Assignment
      ..
    - task: AzureCLI@2
      displayName: Deploy Default Policy Assignments
      ..

- stage: Logging
  displayName: Deploy 
  dependsOn: ManagementGroups
  jobs:
  - job:
    steps:
    - task: AzureCLI@2
      displayName: Az CLI Deploy Logging Resource Group
      ..
    - task: AzureCLI@2
      displayName: Az CLI Deploy Logging
      ..

- stage: NetworkingHub
  displayName: Deploy Hub 
  dependsOn: Logging
  jobs:
  - job:
    steps:
    - task: AzureCLI@2
      displayName: Az CLI Deploy Hub Network Resource Group
      ..
    - task: AzureCLI@2
      displayName: Az CLI Deploy Hub Network
      ..

- stage: NetworkingSpoke
  displayName: Deploy Spoke Networking
  dependsOn: NetworkingHub
  jobs:
  - job:
    steps:
    - task: AzureCLI@2
      displayName: Az CLI Deploy Spoke Network Resource Group
      ..
    - task: AzureCLI@2
      displayName: Az CLI Deploy Spoke Network
      ..

In the next post we’ll continue on this pipeline to add approval steps.

Licensed under CC BY-NC-SA 4.0; code samples licensed under MIT.
comments powered by Disqus
Built with Hugo - Based on Theme Stack designed by Jimmy