In my last post, I discussed the power of the Azure DevOps YAML pipeline with all of its built in features. Today, I would like to focus on a specific use case for the Azure DevOps YAML pipeline with Terraform. By combining these two great technologies, engineers can build repeatable infrastructure in Azure with ease. The following list represents features that Azure DevOps provides by default, which provide key benefits for engineers running Terraform on Azure DevOps YAML pipelines.
- Centralized Reporting – All runs of Terraform will be centrally logged within the Azure DevOps Pipeline Project that it is run from.
- Consistent Behavior – When used with YAML pipelines, the Terraform deploy will be run in the same manner each time and from the same backend hardware dynamically provided by Azure at runtime.
- Environment Variables – Environment variables can easily be passed from the YAML pipeline into Terraform
- Secure – Each pipeline is attached to a Azure Service Connection which is only granted access to the subscription or resource group defined by the administrator
The Azure DevOps Marketplace currently has several options for Terraform integrations into the pipeline from various developers. Regardless of which 3rd party task integration is picked for Terraform and Azure DevOps, the steps and tasks are very similar. We will take a look at what needs to be configured in order to deploy this setup.
Backend State
All of the available Terraform tasks offered within the Azure Marketplace, provide the option for utilizing Azure Blob storage for Terraform state management. This backend also supports state locking and consistency checking via native capabilities of Azure Blob Storage. In order to prepare for this, I have already deployed an Azure Storage account, with a new container named tfstate. I will reference this storage location in my Terraform code dynamically using -backend-config keys. You can delve further into configuring the AzureRM backend through the Terraform documentation here and the Azure documentation here.
Pipeline Configuration
We will again work with the best practice of repeatable code and seek to make separate pipeline yaml files detailing our environment variables, branch triggers, and any other resource dependencies. Then we will need to focus on the yaml template which contains the actual terraform tasks to build infrastructure in Azure.
- task: petergroenewegen.PeterGroenewegen-Xpirit-Vsts-Release-Terraform.Xpirit-Vsts-Release-Terraform.Terraform@2
displayName: 'Terraform plan'
inputs:
TemplatePath: '$(System.DefaultWorkingDirectory)/azure/pipelines
Arguments: 'plan -var arm_subscription_id=$(subscriptionid) -var service_connection_name="$(service_connection_name)" -var cluster_prefix=$(cluster_prefix) -var vm_count=$(vm_count) -var vm_size=$(vm_size) -var $(location) -out=$(System.DefaultWorkingDirectory)/$(cluster_prefix).tfplan'
InstallTerraform: true
Version: 0.12.29
UseAzureSub: true
ConnectedServiceNameARM: '$(service_connection_name)'
ManageState: true
SpecifyStorageAccount: true
StorageAccountResourceGroup: '$(cluster_prefix)-sharedstate'
StorageAccountRM: '$(cluster_prefix)sharedstate'
StorageContainerName: tfstate
InitArguments: '-backend-config="key=$(location).$(cluster_prefix).terraform.tfstate"'
enabled: trueAs part of the plan arguments, we have exported the TFPlan file into the default working directory using:
-out=$(System.DefaultWorkingDirectory)/$(cluster_prefix).tfplanSubsequently, we can create the apply task as the next task within the same yaml template file without needing to define any variables within the arguments section, since the variables have already been defined within the plan step and that information is contained within the tfplan file. For testing purposes, all tasks within the pipeline can be enabled or disabled using the enabled: line. This can be useful while the engineer is testing and verifying the plan before going to the apply task to achieve a successful completion of the pipeline.
Below is the apply task for Terraform Apply:
- task: petergroenewegen.PeterGroenewegen-Xpirit-Vsts-Release-Terraform.Xpirit-Vsts-Release-Terraform.Terraform@2
displayName: 'Terraform apply ENV'
inputs:
TemplatePath: '$(System.DefaultWorkingDirectory)/azure/pipeline
Arguments: apply
PlanPath: '$(cluster_prefix).tfplan'
InstallTerraform: true
Version: 0.12.29
UseAzureSub: true
ConnectedServiceNameARM: '$(service_connection_name)'
ManageState: true
SpecifyStorageAccount: true
StorageAccountResourceGroup: '$(cluster_prefix)-sharedstate'
StorageAccountRM: '$(cluster_prefix_lower)sharedstate'
StorageContainerName: tfstate
InitArguments: '-backend-config="key=centralus.$(cluster_prefix).terraform.tfstate"'We can see that one of the major benefits of deploying Terraform code through the Azure DevOps Yaml pipeline using templates is that the variables are defined in the yaml pipeline file and not the template or the terraform itself. This allows for very extensible code that can be used over and over again with unique variables each time.
Conclusion
Combining Azure DevOps with Terraform is a great benefit for organizations attempting to further advance their Infrastructure as Code processes with the combination of cloud-based backend storage and automation pipelines. If you are interested in exploring how this technology can be deployed in your environment, please reach out to the Azure team at Foghorn and we would be more than happy to assist in building out this automation. In my next post we will cover how to utilize the Azure Key Vault for secrets storage as well as approval gates between pipeline stages with Azure DevOps.
