Tech tutorials Beyond Infrastructure: Using Terraform to Manage Azure Policies
By Insight Editor / 18 Jul 2018 , Updated on 16 May 2019 / Topics: Microsoft Azure
By Insight Editor / 18 Jul 2018 , Updated on 16 May 2019 / Topics: Microsoft Azure
Terraform is a great product for managing Azure infrastructure, but did you know you can do a lot more than just stand up Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) resources?
I was creating a set of Azure policies that I could port across several Azure subscriptions. For simplicity’s sake, we’ll look at a single policy definition around requiring certain tags for every resource in the subscription. Let's see how I used Terraform to quickly accomplish this.
In the past, if you wanted to maintain Azure policies, you could either use the Azure Portal or Azure Resource Manager (ARM) templates.
The Azure Portal is a great tool. However, there’s too much manual intervention and chance for human error when creating and updating policies/assignments.
ARM templates can work as well, but they don't give you the flexibility to see the difference between your configuration before pushing a change. Also, if you want to span multiple subscriptions, you’d have to create your own tooling around managing the changes across all of the subscriptions.
Can Terraform do this more easily?
There are two resources of interest:
Azure tags are key to keeping track of the infrastructure in your subscription. Unless you’ve thoroughly planned out your tagging strategy, you may find yourself in a situation where you want to start requiring a tag on all resources.
Your first question should be, "How compliant is my current infrastructure for this newly required tag?" We can easily answer this with an Azure policy using the audit effect.
Let's take a look at what this definition would look like in Terraform.
We need to set the following parameters:
To get started with the obvious fields, we have:
resource "azurerm_policy_definition" "requiredTag" {
name = "Audit-RequiredTag-Resource"
display_name = "Audit a Required Tag on a Resource"
description = "Audit all resources for a required tag"
policy_type = "Custom"
mode = "All"
policy_rule = "???"
parameters = "???"
}
The policy_rule and parameters must be in the form of JSON. This isn’t due to a design decision on the part the Terraform provider; it’s just how Azure has to interpret the policy. This can be a little convoluted, so let's use the Terraform template_file provider to keep things as clean as possible.
data "template_file" "requiredTag_policy_rule" {
template = <<POLICY_RULE
{
"if": {
"field": "[concat('tags[', parameters('tagName'), ']')]",
"exists": "false"
},
"then": {
"effect": "audit"
}
}
POLICY_RULE
}
data "template_file" "requiredTag_policy_parameters" {
template = <<PARAMETERS
{
"tagName": {
"type": "String",
"metadata": {
"displayName": "Tag Name",
"description": "Name of the tag, such as 'environment'"
}
}
}
PARAMETERS
}
Now we can reference these via interpolation:
resource "azurerm_policy_definition" "requiredTag" {
...
policy_rule = "${data.template_file.requiredTag_policy_rule.rendered}"
parameters = "${data.template_file.requiredTag_policy_parameters.rendered}"
}
Now we’re ready to run a Terraform plan where we end up with something like this:
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ azurerm_policy_definition.requiredTag
id: <computed>
description: "Audit all resources for a required tag"
display_name: "Audit a Required Tag on a Resource"
mode: "All"
name: "Audit-RequiredTag-Resource"
parameters: "{\n \"tagName\": {\n \"type\": \"String\",\n \"metadata\": {\n \"displayName\": \"Tag Name\",\n \"description\": \"Name of the tag, such as 'environment'\"\n }\n }\n}\n"
policy_rule: "{\n \"if\": {\n \"field\": \"[concat('tags[', parameters('tagName'), ']')]\",\n \"exists\": \"false\"\n },\n \"then\": {\n \"effect\": \"audit\"\n }\n}\n"
policy_type: "Custom"
Plan: 1 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------
Running a Terraform apply creates the policy in the Azure subscription. Navigating to the Azure Portal, we can see the custom policy:
Now that we’ve defined a custom Azure policy, we need to assign it to our subscription to make use of it:
azurem_policy_assignment
We need to set the following parameters:
To get started with the obvious fields, we have:
resource "azurerm_policy_assignment" "requiredTag" {
name = "Audit-RequiredTag-${var.requiredTag}"
display_name = "Assign Required Tag '${var.requiredTag}'"
description = "Assignment of Required Tag Policy for '${var.requiredTag}'"
policy_definition_id = "???"
scope = "???"
parameters = "???"
Note the use of a variable 'requiredTag' we’ve created to parameterize the resource creation. More on this in a bit.
This id is simply pulled from the id output from the azurerm_policy_definition resource.
resource "azurerm_policy_assignment" "requiredTag" {
...
policy_definition_id = "${azurerm_policy_definition.requiredTag.id}"
...
}
We want this policy assignment to be for the entire subscription. One option here would be to pass the subscription id as a variable. However, we can source the id from the active Terraform run by using the azurerm_subscription data source.
data "azurerm_subscription" "current" {}
resource "azurerm_policy_assignment" "requiredTag" {
...
scope = "${data.azurerm_subscription.current.id}"
...
}
The last piece we need is the parameters JSON used to assign the 'requireTag' value in the Azure policy. Much like we did before, we leverage the template_file provider.
data "template_file" "requiredTag_policy_assign" {
template = <<PARAMETERS
{
"tagName": {
"value": "${var.requiredTag}"
}
}
PARAMETERS
}
resource "azurerm_policy_assignment" "requiredTag" {
...
parameters = "${data.template_file.requiredTag_policy_assign.rendered}"
}
Now we’re ready to run a Terraform plan where we end up with something like this:
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ azurerm_policy_assignment.policy
id: <computed>
description: "Assignment of Required Tag Policy for 'Environment'"
display_name: "Assign Required Tag Environment"
name: "Audit-RequiredTag-Environment"
parameters: "{\n \"tagName\": {\n \"value\": \"Environment\"\n }\n}\n\n"
policy_definition_id: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/providers/Microsoft.Authorization/policyDefinitions/Audit-RequiredTag-Resource"
scope: "/subscription//subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
Plan: 1 to add, 0 to change, 0 to destroy.
Running a Terraform apply creates the assignment in the Azure subscription. Navigating to the Azure Portal, we can see the assignment:
But what if I have more than one required tag?
One of the key factors to Terraform is the ability to easily scale. Let's modify our current implementation to handle a list of required tags.
First, let's update our variable from a string to a list:
variable "requiredTags" {
default = [
"Environment",
"Owner",
"Department",
]
}
Now we can inject a count parameter in the assignment resource:
resource "azurerm_policy_assignment" "requiredTag" {
count = "${length(var.requiredTags)}"
name = "Audit-RequiredTag-${var.requiredTags[count.index]}"
display_name = "Assign Required Tag '${var.requiredTags[count.index]}'"
description = "Assignment of Required Tag Policy for '${var.requiredTags[count.index]}'"
...
}
Note that we use the length of the requiredTags variable to indicate how many times to repeat the assignment, then index into the list for the name.
The parameters value is a little less clean since we have to inject a different value, depending on the index.
We can do this inline to the assignment without much trouble:
resource "azurerm_policy_assignment" "requiredTag" {
...
parameters = <<PARAMETERS
{
"tagName": {
"value": "${var.requiredTags[count.index]}"
}
}
PARAMETERS
}
Now we’re ready to run a Terraform plan where we end up with something like this:
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ azurerm_policy_assignment.requiredTag[0]
id: <computed>
description: "Assignment of Required Tag Policy for 'Environment'"
display_name: "Assign Required Tag 'Environment'"
name: "Audit-RequiredTag-Environment"
parameters: "{\n \"tagName\": {\n \"value\": \"Environment\"\n }\n}\n\n"
policy_definition_id: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/providers/Microsoft.Authorization/policyDefinitions/Audit-RequiredTag-Resource"
scope: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
+ azurerm_policy_assignment.requiredTag[1]
id: <computed>
description: "Assignment of Required Tag Policy for 'Owner'"
display_name: "Assign Required Tag 'Owner'"
name: "Audit-RequiredTag-Owner"
parameters: "{\n \"tagName\": {\n \"value\": \"Owner\"\n }\n}\n\n"
policy_definition_id: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/providers/Microsoft.Authorization/policyDefinitions/Audit-RequiredTag-Resource"
scope: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
+ azurerm_policy_assignment.requiredTag[2]
id: <computed>
description: "Assignment of Required Tag Policy for 'Department'"
display_name: "Assign Required Tag 'Department'"
name: "Audit-RequiredTag-Department"
parameters: "{\n \"tagName\": {\n \"value\": \"Department\"\n }\n}\n\n"
policy_definition_id: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/providers/Microsoft.Authorization/policyDefinitions/Audit-RequiredTag-Resource"
scope: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
Plan: 3 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------
Notice the tag values are correctly indexed and displayed.
Running a Terraform apply creates the assignment in the Azure subscription. Navigating to the Azure Portal, we can see the assignments:
Once the audit policy assignments have had time to be checked, any non-compliant resources will show up in the portal.
As you can see, I have several resources that don’t have the "Owner" tag, and I can work toward making them compliant.
Once I have a good handle on these required tags, I can update the Terraform from "effect": "audit" to "effect": "deny." This will deny any new request to create or modify any resource that doesn't have the "Owner" tag.
In this article, you’ve been shown how you can leverage Terraform to manage Azure policies to create a consistent governance compliance across your Azure subscription. One really great benefit to this solution is that it can be applied to many different Azure subscriptions without much change in the configuration.
All assets in this article can be found in the GitHub Gist.