Getting Started with Packer and Azure

Published: Sep 8, 2021 by Isaac Johnson

Hashi Packer is the primary tool we use for creating virtual machine images and is one of the few tools at Hashi that is purely OSS (has no “Enterprise” offering). It released 1.0 in 2017 but has versions going back to 2013.

While it is one of the older tools yet to be commercialized (along with Vagrant), it is still actively supported and in 2020 with Packer 1.5 they rolled out HCL2 support.

I noticed a lack of good HCL2 based walk-throughs which inspired this post to take us through creating Linux VMs and images, both VHD and Managed Image based, using Packer.

Let’s get started!

note: if you want to look at the code directly while working through the blog, you can get it from github

Pre-requisites

We wil assume you have an Azure account and the Azure CLI installed. This packer setup actually installs the Azure CLI so you can see the steps in the script:

#!/bin/bash

export DEBIAN_FRONTEND=noninteractive

# Note: Assumes all is being run with root or via sudo

# pre-requisites for cli
apt-get update
apt-get -y install ca-certificates curl apt-transport-https lsb-release gnupg
# install MS key for the repo
curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | tee /etc/apt/trusted.gpg.d/microsoft.gpg > /dev/null
# add the Repo for this Ubuntu release
AZ_REPO=$(lsb_release -cs) && echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" | sudo tee /etc/apt/sources.list.d/azure-cli.list
# update and then install Azure-CLI
apt-get update
apt-get -y install azure-cli

Otherwise, you can follow the official install instructions.

Creating a Packer Linux VM

Login and view your account details. We will need that for the Packer vars later

builder@DESKTOP-QADGF36:~/Workspaces/PackerBlog$ az account list | tail -n15
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "0040020a-b598-444c-8f0b-92f82e1224c0",
    "id": "944c019d-3e46-422b-b63a-86513f147562",
    "isDefault": true,
    "managedByTenants": [],
    "name": "Pay-As-You-Go",
    "state": "Enabled",
    "tenantId": "0040020a-b598-444c-8f0b-92f82e1224c0",
    "user": {
      "name": "isaac.johnson@gmail.com",
      "type": "user"
    }
  }
]

We will then create (or reuse) a service principal.

$ az ad sp create-for-rbac --skip-assignment --name sppackertest
The output includes credentials that you must protect. Be sure that you do not include these credentials in your code or check the credentials into your source control. For more information, see https://aka.ms/azadsp-cli
'name' property in the output is deprecated and will be removed in the future. Use 'appId' instead.
{
  "appId": "d2390854-6661-451d-bee2-3bd9b42b14cc",
  "displayName": "sppackertest",
  "name": "d2390854-6661-451d-bee2-3bd9b42b14cc",
  "password": "bm90IG15IHBhc3N3b3JkCg==",
  "tenant": "0040020a-b598-444c-8f0b-92f82e1224c0"
}

Packer Variables

Next let’s create the packer variables file. The name doesn’t really matter provided we end the file name iwht ‘.pkr.hcl’.

$ cat ubuntu-test.pkr.hcl
variable "client_secret" {
  type =  string
  default = "bm90IG15IHBhc3N3b3JkCg=="
  // Sensitive vars are hidden from output as of Packer v1.6.5
  sensitive = false
}

variable "client_id" {
  type =  string
  default = "d2390854-6661-451d-bee2-3bd9b42b14cc"
  // Sensitive vars are hidden from output as of Packer v1.6.5
  sensitive = false
}

variable "tenant_id" {
  type = string
  default = "0040020a-b598-444c-8f0b-92f82e1224c0"
}

variable "subscription_id" {
  type = string
  default = "944c019d-3e46-422b-b63a-86513f147562"
}

Before we build our JSON, let’s get the current Ubuntu Server available in our region

$ az vm image list-skus --location "East US" --offer UbuntuServer --publisher Canonical -o table
Location    Name
----------  --------------------
eastus      12.04.3-LTS
eastus      12.04.5-LTS
eastus      14.04.0-LTS
eastus      14.04.1-LTS
eastus      14.04.2-LTS
eastus      14.04.3-LTS
eastus      14.04.4-LTS
eastus      14.04.5-DAILY-LTS
eastus      14.04.5-LTS
eastus      16.04-DAILY-LTS
eastus      16.04-LTS
eastus      16.04.0-LTS
eastus      16_04-daily-lts-gen2
eastus      16_04-lts-gen2
eastus      16_04_0-lts-gen2
eastus      18.04-DAILY-LTS
eastus      18.04-LTS
eastus      18.10
eastus      18.10-DAILY
eastus      18_04-daily-lts-gen2
eastus      18_04-lts-gen2
eastus      19.04
eastus      19.04-DAILY
eastus      19.10-DAILY
eastus      19_04-daily-gen2
eastus      19_04-gen2
eastus      19_10-daily-gen2

We will use 18.04-LTS. I tend to like Focal but I do not see 20.04 published by this Publisher just yet (at the time of writing).

Next we need to create the JSON with the build block.

Installing Packer

You can follow the steps here.

Here I will step through installing the latest on my WSL Linux instance.

builder@DESKTOP-QADGF36:~/Workspaces/PackerBlog$ curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
[sudo] password for builder:
OK
builder@DESKTOP-QADGF36:~/Workspaces/PackerBlog$ sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
Get:1 https://apt.releases.hashicorp.com focal InRelease [4419 B]
Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Hit:3 http://archive.ubuntu.com/ubuntu focal InRelease
Get:4 https://apt.releases.hashicorp.com focal/main amd64 Packages [31.2 kB]
Hit:5 https://packages.microsoft.com/repos/azure-cli focal InRelease
Get:6 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Hit:7 https://packages.microsoft.com/repos/microsoft-ubuntu-focal-prod focal InRelease
Hit:8 https://packages.microsoft.com/ubuntu/21.04/prod hirsute InRelease
Get:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]
Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1175 kB]
Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [853 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [24.6 kB]
Get:13 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B]
Fetched 2417 kB in 2s (1404 kB/s)
Reading package lists... Done
builder@DESKTOP-QADGF36:~/Workspaces/PackerBlog$ sudo apt-get update && sudo apt-get install packer
Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease
Hit:2 https://apt.releases.hashicorp.com focal InRelease
Hit:3 http://archive.ubuntu.com/ubuntu focal InRelease
Hit:4 https://packages.microsoft.com/repos/azure-cli focal InRelease
Hit:5 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:6 https://packages.microsoft.com/repos/microsoft-ubuntu-focal-prod focal InRelease
Hit:7 https://packages.microsoft.com/ubuntu/21.04/prod hirsute InRelease
Hit:8 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  packer
0 upgraded, 1 newly installed, 0 to remove and 59 not upgraded.
Need to get 31.7 MB of archives.
After this operation, 145 MB of additional disk space will be used.
Get:1 https://apt.releases.hashicorp.com focal/main amd64 packer amd64 1.7.4 [31.7 MB]
Fetched 31.7 MB in 1s (27.8 MB/s)
Selecting previously unselected package packer.
(Reading database ... 133525 files and directories currently installed.)
Preparing to unpack .../packer_1.7.4_amd64.deb ...
Unpacking packer (1.7.4) ...
Setting up packer (1.7.4) ...

validation

$ packer --version
1.7.4

Packer HCL setup

In the past we would create a JSON file that has all the packer code. And this is still available. However, with HCL2, Packer will look in a directory for all .pkr.hcl files. The names themselves are not important. We could also pack them all into a single file. But it is more manageable to keep them separate.

creating the Vars file

First, we need some variables, most of which we can get from above:

$ cat ubuntu-test-var.pkr.hcl
//  variables.pkr.hcl

// For those variables that you don't provide a default for, you must
// set them from the command line, a var-file, or the environment.

variable "client_secret" {
  type =  string
  default = "bm90IG15IHBhc3N3b3JkCg=="
  // Sensitive vars are hidden from output as of Packer v1.6.5
  sensitive = false
}

variable "client_id" {
  type =  string
  default = "d2390854-6661-451d-bee2-3bd9b42b14cc"
  // Sensitive vars are hidden from output as of Packer v1.6.5
  sensitive = false
}

variable "tenant_id" {
  type = string
  default = "0040020a-b598-444c-8f0b-92f82e1224c0"
}

variable "subscription_id" {
  type = string
  default = "944c019d-3e46-422b-b63a-86513f147562"
}

Creating the VM build file

Next we want to build and provision an instance. We might as well do something interesting, so let us add the Azure CLI to a base Ubuntu image

$ cat ubuntu-test.pkr.hcl

source "azure-arm" "my-example" {
  client_id = "${var.client_id}"
  client_secret = "${var.client_secret}"
  
  resource_group_name = "packerdemo"
  storage_account = "idjvirtualmachines"

  tenant_id= "${var.tenant_id}"
  subscription_id = "${var.subscription_id}"

  capture_container_name = "images"
  capture_name_prefix = "packer"

  os_type = "Linux"
  image_publisher = "Canonical"
  image_offer = "UbuntuServer"
  image_sku = "18.04-LTS"

  location = "East US"
  vm_size = "Standard_DS2_v2"

  azure_tags = {
    dept = "engineering"
  }
}

build {
  sources = ["sources.azure-arm.my-example"]
  
  provisioner "shell-local" {
    execute_command = ["chmod +x {{ .Path }}; {{ .Vars }} sudo -E sh '{{ .Path }}'"]
    inline           = [
      "apt-get update",
      "apt-get upgrade -y",
      "apt-get -y install ca-certificates curl apt-transport-https lsb-release gnupg",
      "curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | tee /etc/apt/trusted.gpg.d/microsoft.gpg > /dev/null",
      "AZ_REPO=$(lsb_release -cs) && echo \"deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main\" | tee /etc/apt/sources.list.d/azure-cli.list",
      "apt-get -y install azure-cli",
      "apt-get update",
      "/usr/sbin/waagent -force -deprovision+user && export HISTSIZE=0 && sync"
    ]
  }
}

note: we will see us fix some of the errors. Some parts above, like using ‘shell-local’ instead of ‘shell’ and how to use a script to affect sudo will be shown. To see the working code, scroll down to ‘Fixing the VHD’. You would still need to handle SP permissions and the pre-reqs listed in the next session

Create necessary prerequisites for Packer run

First, we need to create the resource group refered to in the HCL that will contain our outputs

$ az group create --name packerdemo --location "East US"
{
  "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo",
  "location": "eastus",
  "managedBy": null,
  "name": "packerdemo",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}

Next we should create the storage account for the images.

$ az storage account create -g packerdemo -n idjvirtualmachines --access-tier hot --sku Standard_LRS --location "East US"
{
  "accessTier": "Hot",
  "allowBlobPublicAccess": null,
  "allowCrossTenantReplication": null,
  "allowSharedKeyAccess": null,
  "azureFilesIdentityBasedAuthentication": null,
  "blobRestoreStatus": null,
  "creationTime": "2021-09-06T00:04:35.624306+00:00",
  "customDomain": null,
  "enableHttpsTrafficOnly": true,
  "enableNfsV3": null,
  "encryption": {
    "encryptionIdentity": null,
    "keySource": "Microsoft.Storage",
    "keyVaultProperties": null,
    "requireInfrastructureEncryption": null,
    "services": {
      "blob": {
        "enabled": true,
        "keyType": "Account",
        "lastEnabledTime": "2021-09-06T00:04:35.733670+00:00"
      },
      "file": {
        "enabled": true,
        "keyType": "Account",
        "lastEnabledTime": "2021-09-06T00:04:35.733670+00:00"
      },
      "queue": null,
      "table": null
    }
  },
  "extendedLocation": null,
  "failoverInProgress": null,
  "geoReplicationStats": null,
  "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Storage/storageAccounts/idjvirtualmachines",
  "identity": null,
  "isHnsEnabled": null,
  "keyCreationTime": {
    "key1": "2021-09-06T00:04:35.733670+00:00",
    "key2": "2021-09-06T00:04:35.733670+00:00"
  },
  "keyPolicy": null,
  "kind": "StorageV2",
  "largeFileSharesState": null,
  "lastGeoFailoverTime": null,
  "location": "eastus",
  "minimumTlsVersion": null,
  "name": "idjvirtualmachines",
  "networkRuleSet": {
    "bypass": "AzureServices",
    "defaultAction": "Allow",
    "ipRules": [],
    "resourceAccessRules": null,
    "virtualNetworkRules": []
  },
  "primaryEndpoints": {
    "blob": "https://idjvirtualmachines.blob.core.windows.net/",
    "dfs": "https://idjvirtualmachines.dfs.core.windows.net/",
    "file": "https://idjvirtualmachines.file.core.windows.net/",
    "internetEndpoints": null,
    "microsoftEndpoints": null,
    "queue": "https://idjvirtualmachines.queue.core.windows.net/",
    "table": "https://idjvirtualmachines.table.core.windows.net/",
    "web": "https://idjvirtualmachines.z13.web.core.windows.net/"
  },
  "primaryLocation": "eastus",
  "privateEndpointConnections": [],
  "provisioningState": "Succeeded",
  "resourceGroup": "packerdemo",
  "routingPreference": null,
  "sasPolicy": null,
  "secondaryEndpoints": null,
  "secondaryLocation": null,
  "sku": {
    "name": "Standard_LRS",
    "tier": "Standard"
  },
  "statusOfPrimary": "available",
  "statusOfSecondary": null,
  "tags": {},
  "type": "Microsoft.Storage/storageAccounts"
}

Testing

Now if we try and run this, we will see an error due to permissions.
Packer will attempt to use this service principal to orchestrate the work, however, we merely created the SP and did not grant any access rights:

builder@DESKTOP-QADGF36:~/Workspaces/PackerBlog$ packer build .
azure-arm.my-example: output will be in this color.

==> azure-arm.my-example: Running builder ...
==> azure-arm.my-example: Getting tokens using client secret
Build 'azure-arm.my-example' errored after 315 milliseconds 341 microseconds: adal: Refresh request failed. Status Code = '401'. Response body: {"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret is provided.\r\nTrace ID: 3bd78588-1a80-4fc4-80af-05b4e49c4a00\r\nCorrelation ID: d442a6f4-945d-49eb-a08e-0635d3ebe8a7\r\nTimestamp: 2021-09-05 21:31:37Z","error_codes":[7000215],"timestamp":"2021-09-05 21:31:37Z","trace_id":"3bd78588-1a80-4fc4-80af-05b4e49c4a00","correlation_id":"d442a6f4-945d-49eb-a08e-0635d3ebe8a7","error_uri":"https://login.microsoftonline.com/error?code=7000215"} Endpoint https://login.microsoftonline.com/0040020a-b598-444c-8f0b-92f82e1224c0/oauth2/token?api-version=1.0

==> Wait completed after 315 milliseconds 428 microseconds

==> Some builds didn't complete successfully and had errors:
--> azure-arm.my-example: adal: Refresh request failed. Status Code = '401'. Response body: {"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret is provided.\r\nTrace ID: 3bd78588-1a80-4fc4-80af-05b4e49c4a00\r\nCorrelation ID: d442a6f4-945d-49eb-a08e-0635d3ebe8a7\r\nTimestamp: 2021-09-05 21:31:37Z","error_codes":[7000215],"timestamp":"2021-09-05 21:31:37Z","trace_id":"3bd78588-1a80-4fc4-80af-05b4e49c4a00","correlation_id":"d442a6f4-945d-49eb-a08e-0635d3ebe8a7","error_uri":"https://login.microsoftonline.com/error?code=7000215"} Endpoint https://login.microsoftonline.com/0040020a-b598-444c-8f0b-92f82e1224c0/oauth2/token?api-version=1.0

==> Builds finished but no artifacts were created.

First, we need to find some roles. There are quite a few from which to pick

builder@DESKTOP-QADGF36:~/Workspaces/PackerBlog$ az role definition list --query "[].{name:name, roleType:roleType, roleName:roleName}" --output tsv
8311e382-0749-4cb8-b61a-304f252e45ec    BuiltInRole     AcrPush
312a565d-c81f-4fd8-895a-4e21e48d571c    BuiltInRole     API Management Service Contributor
7f951dda-4ed3-4680-a7ca-43fe172d538d    BuiltInRole     AcrPull
6cef56e8-d556-48e5-a04f-b8e64114680f    BuiltInRole     AcrImageSigner
c2f4ef07-c644-48eb-af81-4b1b4947fb11    BuiltInRole     AcrDelete
cdda3590-29a3-44f6-95f2-9f980659eb04    BuiltInRole     AcrQuarantineReader
c8d4ff99-41c3-41a8-9f60-21dfdad59608    BuiltInRole     AcrQuarantineWriter
e022efe7-f5ba-4159-bbe4-b44f577e9b61    BuiltInRole     API Management Service Operator Role
71522526-b88f-4d52-b57f-d31fc3546d0d    BuiltInRole     API Management Service Reader Role
ae349356-3a1b-4a5e-921d-050484c6347e    BuiltInRole     Application Insights Component Contributor
08954f03-6346-4c2e-81c0-ec3a5cfae23b    BuiltInRole     Application Insights Snapshot Debugger
fd1bd22b-8476-40bc-a0bc-69b95687b9f3    BuiltInRole     Attestation Reader
4fe576fe-1146-4730-92eb-48519fa6bf9f    BuiltInRole     Automation Job Operator
5fb5aef8-1081-4b8e-bb16-9d5d0385bab5    BuiltInRole     Automation Runbook Operator
d3881f73-407a-4167-8283-e981cbba0404    BuiltInRole     Automation Operator
4f8fab4f-1852-4a58-a46a-8eaf358af14a    BuiltInRole     Avere Contributor
c025889f-8102-4ebf-b32c-fc0c6f0c6bd9    BuiltInRole     Avere Operator
0ab0b1a8-8aac-4efd-b8c2-3ee1fb270be8    BuiltInRole     Azure Kubernetes Service Cluster Admin Role
4abbcc35-e782-43d8-92c5-2d3f1bd2253f    BuiltInRole     Azure Kubernetes Service Cluster User Role
423170ca-a8f6-4b0f-8487-9e4eb8f49bfa    BuiltInRole     Azure Maps Data Reader
6f12a6df-dd06-4f3e-bcb1-ce8be600526a    BuiltInRole     Azure Stack Registration Owner
5e467623-bb1f-42f4-a55d-6e525e11384b    BuiltInRole     Backup Contributor
fa23ad8b-c56e-40d8-ac0c-ce449e1d2c64    BuiltInRole     Billing Reader
00c29273-979b-4161-815c-10b084fb9324    BuiltInRole     Backup Operator
a795c7a0-d4a2-40c1-ae25-d81f01202912    BuiltInRole     Backup Reader
31a002a1-acaf-453e-8a5b-297c9ca1ea24    BuiltInRole     Blockchain Member Node Access (Preview)
5e3c6656-6cfa-4708-81fe-0de47ac73342    BuiltInRole     BizTalk Contributor
426e0c7f-0c7e-4658-b36f-ff54d6c29b45    BuiltInRole     CDN Endpoint Contributor
871e35f6-b5c1-49cc-a043-bde969a0f2cd    BuiltInRole     CDN Endpoint Reader
ec156ff8-a8d1-4d15-830c-5b80698ca432    BuiltInRole     CDN Profile Contributor
8f96442b-4075-438f-813d-ad51ab4019af    BuiltInRole     CDN Profile Reader
b34d265f-36f7-4a0d-a4d4-e158ca92e90f    BuiltInRole     Classic Network Contributor
86e8f5dc-a6e9-4c67-9d15-de283e8eac25    BuiltInRole     Classic Storage Account Contributor
985d6b00-f706-48f5-a6fe-d0ca12fb668d    BuiltInRole     Classic Storage Account Key Operator Service Role
9106cda0-8a86-4e81-b686-29a22c54effe    BuiltInRole     ClearDB MySQL DB Contributor
d73bb868-a0df-4d4d-bd69-98a00b01fccb    BuiltInRole     Classic Virtual Machine Contributor
a97b65f3-24c7-4388-baec-2e87135dc908    BuiltInRole     Cognitive Services User
b59867f0-fa02-499b-be73-45a86b5b3e1c    BuiltInRole     Cognitive Services Data Reader (Preview)
25fbc0a9-bd7c-42a3-aa1a-3b75d497ee68    BuiltInRole     Cognitive Services Contributor
db7b14f2-5adf-42da-9f96-f2ee17bab5cb    BuiltInRole     CosmosBackupOperator
b24988ac-6180-42a0-ab88-20f7382dd24c    BuiltInRole     Contributor
fbdf93bf-df7d-467e-a4d2-9458aa1360c8    BuiltInRole     Cosmos DB Account Reader Role
434105ed-43f6-45c7-a02f-909b2ba83430    BuiltInRole     Cost Management Contributor
72fafb9e-0641-4937-9268-a91bfd8191a3    BuiltInRole     Cost Management Reader
add466c9-e687-43fc-8d98-dfcf8d720be5    BuiltInRole     Data Box Contributor
028f4ed7-e2a9-465e-a8f4-9c0ffdfdc027    BuiltInRole     Data Box Reader
673868aa-7521-48a0-acc6-0f60742d39f5    BuiltInRole     Data Factory Contributor
150f5e0c-0603-4f03-8c7f-cf70034c4e90    BuiltInRole     Data Purger
47b7735b-770e-4598-a7da-8b91488b4c88    BuiltInRole     Data Lake Analytics Developer
76283e04-6283-4c54-8f91-bcf1374a3c64    BuiltInRole     DevTest Labs User
5bd9cd88-fe45-4216-938b-f97437e15450    BuiltInRole     DocumentDB Account Contributor
befefa01-2a29-4197-83a8-272ff33ce314    BuiltInRole     DNS Zone Contributor
428e0ff0-5e57-4d9c-a221-2c70d0e0a443    BuiltInRole     EventGrid EventSubscription Contributor
2414bbcf-6497-4faf-8c65-045460748405    BuiltInRole     EventGrid EventSubscription Reader
b60367af-1334-4454-b71e-769d9a4f83d9    BuiltInRole     Graph Owner
8d8d5a11-05d3-4bda-a417-a08778121c7c    BuiltInRole     HDInsight Domain Services Contributor
03a6d094-3444-4b3d-88af-7477090a9e5e    BuiltInRole     Intelligent Systems Account Contributor
f25e0fa2-a7c8-4377-a976-54943a77a395    BuiltInRole     Key Vault Contributor
ee361c5d-f7b5-4119-b4b6-892157c8f64c    BuiltInRole     Knowledge Consumer
b97fb8bc-a8b2-4522-a38b-dd33c7e65ead    BuiltInRole     Lab Creator
73c42c96-874c-492b-b04d-ab87d138a893    BuiltInRole     Log Analytics Reader
92aaf0da-9dab-42b6-94a3-d43ce8d16293    BuiltInRole     Log Analytics Contributor
515c2055-d9d4-4321-b1b9-bd0c9a0f79fe    BuiltInRole     Logic App Operator
87a39d53-fc1b-424a-814c-f7e04687dc9e    BuiltInRole     Logic App Contributor
c7393b34-138c-406f-901b-d8cf2b17e6ae    BuiltInRole     Managed Application Operator Role
b9331d33-8a36-4f8c-b097-4f54124fdb44    BuiltInRole     Managed Applications Reader
f1a07417-d97a-45cb-824c-7a7467783830    BuiltInRole     Managed Identity Operator
e40ec5ca-96e0-45a2-b4ff-59039f2c2b59    BuiltInRole     Managed Identity Contributor
5d58bcaf-24a5-4b20-bdb6-eed9f69fbe4c    BuiltInRole     Management Group Contributor
ac63b705-f282-497d-ac71-919bf39d939d    BuiltInRole     Management Group Reader
3913510d-42f4-4e42-8a64-420c390055eb    BuiltInRole     Monitoring Metrics Publisher
43d0d8ad-25c7-4714-9337-8ba259a9fe05    BuiltInRole     Monitoring Reader
4d97b98b-1d4f-4787-a291-c67834d212e7    BuiltInRole     Network Contributor
749f88d5-cbae-40b8-bcfc-e573ddc772fa    BuiltInRole     Monitoring Contributor
5d28c62d-5b37-4476-8438-e587778df237    BuiltInRole     New Relic APM Account Contributor
8e3af657-a8ff-443c-a75c-2fe8c4bcb635    BuiltInRole     Owner
acdd72a7-3385-48ef-bd42-f606fba81ae7    BuiltInRole     Reader
e0f68234-74aa-48ed-b826-c38b57376e17    BuiltInRole     Redis Cache Contributor
c12c1c16-33a1-487b-954d-41c89c60f349    BuiltInRole     Reader and Data Access
36243c78-bf99-498c-9df9-86d9f8d28608    BuiltInRole     Resource Policy Contributor
188a0f2f-5c9e-469b-ae67-2aa5ce574b94    BuiltInRole     Scheduler Job Collections Contributor
7ca78c08-252a-4471-8644-bb5ff32d4ba0    BuiltInRole     Search Service Contributor
fb1c8493-542b-48eb-b624-b4c8fea62acd    BuiltInRole     Security Admin
e3d13bf0-dd5a-482e-ba6b-9b8433878d10    BuiltInRole     Security Manager (Legacy)
39bc4728-0917-49c7-9d2c-d95423bc2eb4    BuiltInRole     Security Reader
8bbe83f1-e2a6-4df7-8cb4-4e04d4e5c827    BuiltInRole     Spatial Anchors Account Contributor
6670b86e-a3f7-4917-ac9b-5d6ab1be4567    BuiltInRole     Site Recovery Contributor
494ae006-db33-4328-bf46-533a6560a3ca    BuiltInRole     Site Recovery Operator
5d51204f-eb77-4b1c-b86a-2ec626c49413    BuiltInRole     Spatial Anchors Account Reader
dbaa88c4-0c30-4179-9fb3-46319faa6149    BuiltInRole     Site Recovery Reader
70bbe301-9835-447d-afdd-19eb3167307c    BuiltInRole     Spatial Anchors Account Owner
4939a1f6-9ae0-4e48-a1e0-f2cbe897382d    BuiltInRole     SQL Managed Instance Contributor
9b7fa17d-e63e-47b0-bb0a-15c516ac86ec    BuiltInRole     SQL DB Contributor
056cd41c-7e88-42e1-933e-88ba6a50c9c3    BuiltInRole     SQL Security Manager
17d1049b-9a84-46fb-8f53-869881c3d3ab    BuiltInRole     Storage Account Contributor
6d8ee4ec-f05a-4a1d-8b00-a9b17e38b437    BuiltInRole     SQL Server Contributor
81a9662b-bebf-436f-a333-f67b29880f12    BuiltInRole     Storage Account Key Operator Service Role
ba92f5b4-2d11-453d-a403-e96b0029c9fe    BuiltInRole     Storage Blob Data Contributor
b7e6dc6d-f1e8-4753-8033-0f276bb0955b    BuiltInRole     Storage Blob Data Owner
2a2b9908-6ea1-4ae2-8e65-a410df84e7d1    BuiltInRole     Storage Blob Data Reader
974c5e8b-45b9-4653-ba55-5f855dd0fb88    BuiltInRole     Storage Queue Data Contributor
8a0f0c08-91a1-4084-bc3d-661d67233fed    BuiltInRole     Storage Queue Data Message Processor
c6a89b2d-59bc-44d0-9896-0f6e12d7b80a    BuiltInRole     Storage Queue Data Message Sender
19e7f393-937e-4f77-808e-94535e297925    BuiltInRole     Storage Queue Data Reader
cfd33db0-3dd1-45e3-aa9d-cdbdf3b6f24e    BuiltInRole     Support Request Contributor
a4b10055-b0c7-44c2-b00f-c7b5b3550cf7    BuiltInRole     Traffic Manager Contributor
1c0163c0-47e6-4577-8991-ea5c82e286e4    BuiltInRole     Virtual Machine Administrator Login
18d7d88d-d35e-4fb5-a5c3-7773c20a72d9    BuiltInRole     User Access Administrator
fb879df8-f326-4884-b1cf-06f3ad86be52    BuiltInRole     Virtual Machine User Login
9980e02c-c2be-4d73-94e8-173b1dc7cf3c    BuiltInRole     Virtual Machine Contributor
2cc479cb-7b4d-49a8-b449-8c00fd0f0a4b    BuiltInRole     Web Plan Contributor
de139f84-1756-47ae-9be6-808fbbe84772    BuiltInRole     Website Contributor
090c5cfd-751d-490a-894a-3ce6f1109419    BuiltInRole     Azure Service Bus Data Owner
f526a384-b230-433a-b45c-95f59c4a2dec    BuiltInRole     Azure Event Hubs Data Owner
bbf86eb8-f7b4-4cce-96e4-18cddf81d86e    BuiltInRole     Attestation Contributor
61ed4efc-fab3-44fd-b111-e24485cc132a    BuiltInRole     HDInsight Cluster Operator
230815da-be43-4aae-9cb4-875f7bd000aa    BuiltInRole     Cosmos DB Operator
48b40c6e-82e0-4eb3-90d5-19e40f49b624    BuiltInRole     Hybrid Server Resource Administrator
5d1e5ee4-7c68-4a71-ac8b-0739630a3dfb    BuiltInRole     Hybrid Server Onboarding
a638d3c7-ab3a-418d-83e6-5f17a39d4fde    BuiltInRole     Azure Event Hubs Data Receiver
2b629674-e913-4c01-ae53-ef4638d8f975    BuiltInRole     Azure Event Hubs Data Sender
4f6d3b9b-027b-4f4c-9142-0e5a2a2247e0    BuiltInRole     Azure Service Bus Data Receiver
69a216fc-b8fb-44d8-bc22-1f3c2cd27a39    BuiltInRole     Azure Service Bus Data Sender
aba4ae5f-2193-4029-9191-0cb91df5e314    BuiltInRole     Storage File Data SMB Share Reader
0c867c2a-1d8c-454a-a3db-ab2ea1bdc8bb    BuiltInRole     Storage File Data SMB Share Contributor
b12aa53e-6015-4669-85d0-8515ebb3ae7f    BuiltInRole     Private DNS Zone Contributor
db58b8e5-c6ad-4a2a-8342-4190687cbf4a    BuiltInRole     Storage Blob Delegator
1d18fff3-a72a-46b5-b4a9-0b38a3cd7e63    BuiltInRole     Desktop Virtualization User
a7264617-510b-434b-a828-9731dc254ea7    BuiltInRole     Storage File Data SMB Share Elevated Contributor
41077137-e803-4205-871c-5a86e6a753b4    BuiltInRole     Blueprint Contributor
437d2ced-4a38-4302-8479-ed2bcb43d090    BuiltInRole     Blueprint Operator
ab8e14d6-4a74-4a29-9ba8-549422addade    BuiltInRole     Azure Sentinel Contributor
3e150937-b8fe-4cfb-8069-0eaf05ecd056    BuiltInRole     Azure Sentinel Responder
8d289c81-5878-46d4-8554-54e1e3d8b5cb    BuiltInRole     Azure Sentinel Reader
b279062a-9be3-42a0-92ae-8b3cf002ec4d    BuiltInRole     Workbook Reader
e8ddcd69-c73f-4f9f-9844-4100522f16ad    BuiltInRole     Workbook Contributor
66bb4e9e-b016-4a94-8249-4c0511c2be84    BuiltInRole     Policy Insights Data Writer (Preview)
04165923-9d83-45d5-8227-78b77b0a687e    BuiltInRole     SignalR AccessKey Reader
8cf5e20a-e4b2-4e9d-b3a1-5ceb692c2761    BuiltInRole     SignalR Contributor
b64e21ea-ac4e-4cdf-9dc9-5b892992bee7    BuiltInRole     Azure Connected Machine Onboarding
cd570a14-e51a-42ad-bac8-bafd67325302    BuiltInRole     Azure Connected Machine Resource Administrator
91c1777a-f3dc-4fae-b103-61d183457e46    BuiltInRole     Managed Services Registration assignment Delete Role
5ae67dd6-50cb-40e7-96ff-dc2bfa4b606b    BuiltInRole     App Configuration Data Owner
516239f1-63e1-4d78-a4de-a74fb236a071    BuiltInRole     App Configuration Data Reader
34e09817-6cbe-4d01-b1a2-e0eac5743d41    BuiltInRole     Kubernetes Cluster - Azure Arc Onboarding
7f646f1b-fa08-80eb-a22b-edd6ce5c915c    BuiltInRole     Experimentation Contributor
466ccd10-b268-4a11-b098-b4849f024126    BuiltInRole     Cognitive Services QnA Maker Reader
f4cc2bf9-21be-47a1-bdf1-5c5804381025    BuiltInRole     Cognitive Services QnA Maker Editor
7f646f1b-fa08-80eb-a33b-edd6ce5c915c    BuiltInRole     Experimentation Administrator
3df8b902-2a6f-47c7-8cc5-360e9b272a7e    BuiltInRole     Remote Rendering Administrator
d39065c4-c120-43c9-ab0a-63eed9795f0a    BuiltInRole     Remote Rendering Client
641177b8-a67a-45b9-a033-47bc880bb21e    BuiltInRole     Managed Application Contributor Role
612c2aa1-cb24-443b-ac28-3ab7272de6f5    BuiltInRole     Security Assessment Contributor
4a9ae827-6dc8-4573-8ac7-8239d42aa03f    BuiltInRole     Tag Contributor
c7aa55d3-1abb-444a-a5ca-5e51e485d6ec    BuiltInRole     Integration Service Environment Developer
a41e2c5b-bd99-4a07-88f4-9bf657a760b8    BuiltInRole     Integration Service Environment Contributor
ed7f3fbd-7b88-4dd4-9017-9adb7ce333f8    BuiltInRole     Azure Kubernetes Service Contributor Role
d57506d4-4c8d-48b1-8587-93c323f6a5a3    BuiltInRole     Azure Digital Twins Data Reader
bcd981a7-7f74-457b-83e1-cceb9e632ffe    BuiltInRole     Azure Digital Twins Data Owner
350f8d15-c687-4448-8ae1-157740a3936d    BuiltInRole     Hierarchy Settings Administrator
5a1fc7df-4bf1-4951-a576-89034ee01acd    BuiltInRole     FHIR Data Contributor
3db33094-8700-4567-8da5-1501d4e7e843    BuiltInRole     FHIR Data Exporter
4c8d0bbc-75d3-4935-991f-5f3c56d81508    BuiltInRole     FHIR Data Reader
3f88fce4-5892-4214-ae73-ba5294559913    BuiltInRole     FHIR Data Writer
49632ef5-d9ac-41f4-b8e7-bbe587fa74a1    BuiltInRole     Experimentation Reader
4dd61c23-6743-42fe-a388-d8bdd41cb745    BuiltInRole     Object Understanding Account Owner
8f5e0ce6-4f7b-4dcf-bddf-e6f48634a204    BuiltInRole     Azure Maps Data Contributor
c1ff6cc2-c111-46fe-8896-e0ef812ad9f3    BuiltInRole     Cognitive Services Custom Vision Contributor
5c4089e1-6d96-4d2f-b296-c1bc7137275f    BuiltInRole     Cognitive Services Custom Vision Deployment
88424f51-ebe7-446f-bc41-7fa16989e96c    BuiltInRole     Cognitive Services Custom Vision Labeler
93586559-c37d-4a6b-ba08-b9f0940c2d73    BuiltInRole     Cognitive Services Custom Vision Reader
0a5ae4ab-0d65-4eeb-be61-29fc9b54394b    BuiltInRole     Cognitive Services Custom Vision Trainer
00482a5a-887f-4fb3-b363-3b7fe8e74483    BuiltInRole     Key Vault Administrator
14b46e9e-c2b7-41b4-b07b-48a6ebf60603    BuiltInRole     Key Vault Crypto Officer
12338af0-0e69-4776-bea7-57ae8d297424    BuiltInRole     Key Vault Crypto User
b86a8fe4-44ce-4948-aee5-eccb2c155cd7    BuiltInRole     Key Vault Secrets Officer
4633458b-17de-408a-b874-0445c86b69e6    BuiltInRole     Key Vault Secrets User
a4417e6f-fecd-4de8-b567-7b0420556985    BuiltInRole     Key Vault Certificates Officer
21090545-7ca7-4776-b22c-e363652d74d2    BuiltInRole     Key Vault Reader
e147488a-f6f5-4113-8e2d-b22465e65bf6    BuiltInRole     Key Vault Crypto Service Encryption User
63f0a09d-1495-4db4-a681-037d84835eb4    BuiltInRole     Azure Arc Kubernetes Viewer
5b999177-9696-4545-85c7-50de3797e5a1    BuiltInRole     Azure Arc Kubernetes Writer
8393591c-06b9-48a2-a542-1bd6b377f6a2    BuiltInRole     Azure Arc Kubernetes Cluster Admin
dffb1e0c-446f-4dde-a09f-99eb5cc68b96    BuiltInRole     Azure Arc Kubernetes Admin
b1ff04bb-8a4e-4dc4-8eb5-8693973ce19b    BuiltInRole     Azure Kubernetes Service RBAC Cluster Admin
3498e952-d568-435e-9b2c-8d77e338d7f7    BuiltInRole     Azure Kubernetes Service RBAC Admin
7f6c6a51-bcf8-42ba-9220-52d62157d7db    BuiltInRole     Azure Kubernetes Service RBAC Reader
a7ffa36f-339b-4b5c-8bdf-e2c188b2c0eb    BuiltInRole     Azure Kubernetes Service RBAC Writer
82200a5b-e217-47a5-b665-6d8765ee745b    BuiltInRole     Services Hub Operator
d18777c0-1514-4662-8490-608db7d334b6    BuiltInRole     Object Understanding Account Reader
00493d72-78f6-4148-b6c5-d3ce8e4799dd    BuiltInRole     Azure Arc Enabled Kubernetes Cluster User Role
420fcaa2-552c-430f-98ca-3264be4806c7    BuiltInRole     SignalR App Server (Preview)
fd53cd77-2268-407a-8f46-7e7863d0f521    BuiltInRole     SignalR Serverless Contributor (Preview)
daa9e50b-21df-454c-94a6-a8050adab352    BuiltInRole     Collaborative Data Contributor
e9dba6fb-3d52-4cf0-bce3-f06ce71b9e0f    BuiltInRole     Device Update Reader
02ca0879-e8e4-47a5-a61e-5c618b76e64a    BuiltInRole     Device Update Administrator
0378884a-3af5-44ab-8323-f5b22f9f3c98    BuiltInRole     Device Update Content Administrator
e4237640-0e3d-4a46-8fda-70bc94856432    BuiltInRole     Device Update Deployments Administrator
49e2f5d2-7741-4835-8efa-19e1fe35e47f    BuiltInRole     Device Update Deployments Reader
d1ee9a80-8b14-47f0-bdc2-f4a351625a7b    BuiltInRole     Device Update Content Reader
cb43c632-a144-4ec5-977c-e80c4affc34a    BuiltInRole     Cognitive Services Metrics Advisor Administrator
3b20f47b-3825-43cb-8114-4bd2201156a8    BuiltInRole     Cognitive Services Metrics Advisor User
2c56ea50-c6b3-40a6-83c0-9d98858bc7d2    BuiltInRole     Schema Registry Reader (Preview)
5dffeca3-4936-4216-b2bc-10343a5abb25    BuiltInRole     Schema Registry Contributor (Preview)
7ec7ccdc-f61e-41fe-9aaf-980df0a44eba    BuiltInRole     AgFood Platform Service Reader
8508508a-4469-4e45-963b-2518ee0bb728    BuiltInRole     AgFood Platform Service Contributor
f8da80de-1ff9-4747-ad80-a19b7f6079e3    BuiltInRole     AgFood Platform Service Admin
18500a29-7fe2-46b2-a342-b16a415e101d    BuiltInRole     Managed HSM contributor
0b555d9b-b4a7-4f43-b330-627f0e5be8f0    BuiltInRole     Security Detonation Chamber Submitter
ddde6b66-c0df-4114-a159-3618637b3035    BuiltInRole     SignalR Service Reader (Preview)
7e4f1700-ea5a-4f59-8f37-079cfe29dce3    BuiltInRole     SignalR Service Owner
f7b75c60-3036-4b75-91c3-6b41c27c1689    BuiltInRole     Reservation Purchaser
635dd51f-9968-44d3-b7fb-6d9a6bd613ae    BuiltInRole     AzureML Metrics Writer (preview)
e5e2a7ff-d759-4cd2-bb51-3152d37e2eb1    BuiltInRole     Storage Account Backup Contributor Role
6188b7c9-7d01-4f99-a59f-c88b630326c0    BuiltInRole     Experimentation Metric Contributor
9ef4ef9c-a049-46b0-82ab-dd8ac094c889    BuiltInRole     Project Babylon Data Curator
c8d896ba-346d-4f50-bc1d-7d1c84130446    BuiltInRole     Project Babylon Data Reader
05b7651b-dc44-475e-b74d-df3db49fae0f    BuiltInRole     Project Babylon Data Source Administrator
8a3c2885-9b38-4fd2-9d99-91af537c1347    BuiltInRole     Purview Data Curator
ff100721-1b9d-43d8-af52-42b69c1272db    BuiltInRole     Purview Data Reader
200bba9e-f0c8-430f-892b-6f0794863803    BuiltInRole     Purview Data Source Administrator
ca6382a4-1721-4bcf-a114-ff0c70227b6b    BuiltInRole     Application Group Contributor
49a72310-ab8d-41df-bbb0-79b649203868    BuiltInRole     Desktop Virtualization Reader
082f0a83-3be5-4ba1-904c-961cca79b387    BuiltInRole     Desktop Virtualization Contributor
21efdde3-836f-432b-bf3d-3e8e734d4b2b    BuiltInRole     Desktop Virtualization Workspace Contributor
ea4bfff8-7fb4-485a-aadd-d4129a0ffaa6    BuiltInRole     Desktop Virtualization User Session Operator
2ad6aaab-ead9-4eaa-8ac5-da422f562408    BuiltInRole     Desktop Virtualization Session Host Operator
ceadfde2-b300-400a-ab7b-6143895aa822    BuiltInRole     Desktop Virtualization Host Pool Reader
e307426c-f9b6-4e81-87de-d99efb3c32bc    BuiltInRole     Desktop Virtualization Host Pool Contributor
aebf23d0-b568-4e86-b8f9-fe83a2c6ab55    BuiltInRole     Desktop Virtualization Application Group Reader
86240b0e-9422-4c43-887b-b61143f32ba8    BuiltInRole     Desktop Virtualization Application Group Contributor
0fa44ee9-7a7d-466b-9bb2-2bf446b1204d    BuiltInRole     Desktop Virtualization Workspace Reader
3e5e47e6-65f7-47ef-90b5-e5dd4d455f24    BuiltInRole     Disk Backup Reader
b8b15564-4fa6-4a59-ab12-03e1d9594795    BuiltInRole     Autonomous Development Platform Data Contributor (Preview)
d63b75f7-47ea-4f27-92ac-e0d173aaf093    BuiltInRole     Autonomous Development Platform Data Reader (Preview)
27f8b550-c507-4db9-86f2-f4b8e816d59d    BuiltInRole     Autonomous Development Platform Data Owner (Preview)
b50d9833-a0cb-478e-945f-707fcc997c13    BuiltInRole     Disk Restore Operator
7efff54f-a5b4-42b5-a1c5-5411624893ce    BuiltInRole     Disk Snapshot Contributor
5548b2cf-c94c-4228-90ba-30851930a12f    BuiltInRole     Microsoft.Kubernetes connected cluster role
a37b566d-3efa-4beb-a2f2-698963fa42ce    BuiltInRole     Security Detonation Chamber Submission Manager
352470b3-6a9c-4686-b503-35deb827e500    BuiltInRole     Security Detonation Chamber Publisher
7a6f0e70-c033-4fb1-828c-08514e5f4102    BuiltInRole     Collaborative Runtime Operator
5432c526-bc82-444a-b7ba-57c5b0b5b34f    BuiltInRole     CosmosRestoreOperator
a1705bd2-3a8f-45a5-8683-466fcfd5cc24    BuiltInRole     FHIR Data Converter
f4c81013-99ee-4d62-a7ee-b3f1f648599a    BuiltInRole     Azure Sentinel Automation Contributor
0e5f05e5-9ab9-446b-b98d-1e2157c94125    BuiltInRole     Quota Request Operator
1e241071-0855-49ea-94dc-649edcd759de    BuiltInRole     EventGrid Contributor
28241645-39f8-410b-ad48-87863e2951d5    BuiltInRole     Security Detonation Chamber Reader
4a167cdf-cb95-4554-9203-2347fe489bd9    BuiltInRole     Object Anchors Account Reader
ca0835dd-bacc-42dd-8ed2-ed5e7230d15b    BuiltInRole     Object Anchors Account Owner
d17ce0a2-0697-43bc-aac5-9113337ab61c    BuiltInRole     WorkloadBuilder Migration Agent Role
12cf5a90-567b-43ae-8102-96cf46c7d9b4    BuiltInRole     Web PubSub Service Owner (Preview)
bfb1c7d2-fb1a-466b-b2ba-aee63b92deaf    BuiltInRole     Web PubSub Service Reader (Preview)
b5537268-8956-4941-a8f0-646150406f0c    BuiltInRole     Azure Spring Cloud Data Reader
f2dc8367-1007-4938-bd23-fe263f013447    BuiltInRole     Cognitive Services Speech User
0e75ca1e-0464-4b4d-8b93-68208a576181    BuiltInRole     Cognitive Services Speech Contributor
9894cab4-e18a-44aa-828b-cb588cd6f2d7    BuiltInRole     Cognitive Services Face Recognizer
054126f8-9a2b-4f1c-a9ad-eca461f08466    BuiltInRole     Media Services Account Administrator
532bc159-b25e-42c0-969e-a1d439f60d77    BuiltInRole     Media Services Live Events Administrator
e4395492-1534-4db2-bedf-88c14621589c    BuiltInRole     Media Services Media Operator
c4bba371-dacd-4a26-b320-7250bca963ae    BuiltInRole     Media Services Policy Administrator
99dba123-b5fe-44d5-874c-ced7199a5804    BuiltInRole     Media Services Streaming Endpoints Administrator
1ec5b3c1-b17e-4e25-8312-2acb3c3c5abf    BuiltInRole     Stream Analytics Query Tester
a2138dac-4907-4679-a376-736901ed8ad8    BuiltInRole     AnyBuild Builder
b447c946-2db7-41ec-983d-d8bf3b1c77e3    BuiltInRole     IoT Hub Data Reader
494bdba2-168f-4f31-a0a1-191d2f7c028c    BuiltInRole     IoT Hub Twin Contributor
4ea46cd5-c1b2-4a8e-910b-273211f9ce47    BuiltInRole     IoT Hub Registry Contributor
4fc6c259-987e-4a07-842e-c321cc9d413f    BuiltInRole     IoT Hub Data Contributor
15e0f5a1-3450-4248-8e25-e2afe88a9e85    BuiltInRole     Test Base Reader
1407120a-92aa-4202-b7e9-c0e197c71c8f    BuiltInRole     Search Index Data Reader
8ebe5a00-799e-43f5-93ac-243d3dce84a7    BuiltInRole     Search Index Data Contributor
76199698-9eea-4c19-bc75-cec21354c6b6    BuiltInRole     Storage Table Data Reader
0a9a7e1f-b9d0-4cc4-a60d-0319b160aaa3    BuiltInRole     Storage Table Data Contributor
e89c7a3c-2f64-4fa1-a847-3e4c9ba4283a    BuiltInRole     DICOM Data Reader
58a3b984-7adf-4c20-983a-32417c86fbc8    BuiltInRole     DICOM Data Owner
d5a91429-5739-47e2-a06b-3470a27159e7    BuiltInRole     EventGrid Data Sender
f6c7c914-8db3-469d-8ca1-694a8f32e121    BuiltInRole     AzureML Data Scientist
22926164-76b3-42b3-bc55-97df8dab3e41    BuiltInRole     Grafana Admin
e8113dce-c529-4d33-91fa-e9b972617508    BuiltInRole     Azure Connected SQL Server Onboarding
26baccc8-eea7-41f1-98f4-1762cc7f685d    BuiltInRole     Azure Relay Sender
2787bf04-f1f5-4bfe-8383-c8a24483ee38    BuiltInRole     Azure Relay Owner
26e0b698-aa6d-4085-9386-aadae190014d    BuiltInRole     Azure Relay Listener
60921a7e-fef1-4a43-9b16-a26c52ad4769    BuiltInRole     Grafana Viewer
a79a5197-3a5c-4973-a920-486035ffd60f    BuiltInRole     Grafana Editor
f353d9bd-d4a6-484e-a77a-8050b599b867    BuiltInRole     Automation Contributor
85cb6faf-e071-4c9b-8136-154b5a04f717    BuiltInRole     Kubernetes Extension Contributor
10745317-c249-44a1-a5ce-3a4353c0bbd8    BuiltInRole     Device Provisioning Service Data Reader
dfce44e4-17b7-4bd1-a6d1-04996ec95633    BuiltInRole     Device Provisioning Service Data Contributor
2837e146-70d7-4cfd-ad55-7efa6464f958    BuiltInRole     CodeSigning Certificate Profile Signer
cff1b556-2399-4e7e-856d-a8f754be7b65    BuiltInRole     Azure Spring Cloud Service Registry Reader
f5880b48-c26d-48be-b172-7927bfa1c8f1    BuiltInRole     Azure Spring Cloud Service Registry Contributor
d04c6db6-4947-4782-9e91-30a88feb7be7    BuiltInRole     Azure Spring Cloud Config Server Reader

If we look at the documentation we can see how we can really tailor the role assignment.

For the demo, however, i’ll just grant this SP Contributor rights to the subscription (since it needs to create Resource Groups).

$ az role assignment create --assignee "d2390854-6661-451d-bee2-3bd9b42b14cc" --role "Contributor" --subscription "944c019d-3e46-422b-b63a-86513f147562"
{
  "canDelegate": null,
  "condition": null,
  "conditionVersion": null,
  "description": null,
  "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/providers/Microsoft.Authorization/roleAssignments/e6a3bc21-86cf-47b9-ba20-9dab79388850",
  "name": "e6a3bc21-86cf-47b9-ba20-9dab79388850",
  "principalId": "7e011177-1825-4e97-ac5c-72d5e7361147",
  "principalType": "ServicePrincipal",
  "roleDefinitionId": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c",
  "scope": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562",
  "type": "Microsoft.Authorization/roleAssignments"
}

Now when we try, it should work (though it will yell at use for using VHD still)

builder@DESKTOP-QADGF36:~/Workspaces/PackerBlog$ packer build .
azure-arm.my-example: output will be in this color.

==> azure-arm.my-example: Running builder ...
==> azure-arm.my-example: Getting tokens using client secret
==> azure-arm.my-example: Getting tokens using client secret
    azure-arm.my-example: Creating Azure Resource Manager (ARM) client ...
==> azure-arm.my-example: Warning: You are using Azure Packer Builder to create VHDs which is being deprecated, consider using Managed Images. Learn more https://www.packer.io/docs/builders/azure/arm#azure-arm-builder-specific-options
==> azure-arm.my-example: WARNING: Zone resiliency may not be supported in East US, checkout the docs at https://docs.microsoft.com/en-us/azure/availability-zones/
==> azure-arm.my-example: Creating resource group ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-gsdf8clejz'
==> azure-arm.my-example:  -> Location          : 'East US'
==> azure-arm.my-example:  -> Tags              :
==> azure-arm.my-example:  ->> dept : engineering
==> azure-arm.my-example: Validating deployment template ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-gsdf8clejz'
==> azure-arm.my-example:  -> DeploymentName    : 'pkrdpgsdf8clejz'
==> azure-arm.my-example: Deploying deployment template ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-gsdf8clejz'
==> azure-arm.my-example:  -> DeploymentName    : 'pkrdpgsdf8clejz'
==> azure-arm.my-example:
==> azure-arm.my-example: Getting the VM's IP address ...
==> azure-arm.my-example:  -> ResourceGroupName   : 'pkr-Resource-Group-gsdf8clejz'
==> azure-arm.my-example:  -> PublicIPAddressName : 'pkripgsdf8clejz'
==> azure-arm.my-example:  -> NicName             : 'pkrnigsdf8clejz'
==> azure-arm.my-example:  -> Network Connection  : 'PublicEndpoint'
==> azure-arm.my-example:  -> IP Address          : '104.211.48.163'
==> azure-arm.my-example: Waiting for SSH to become available...
==> azure-arm.my-example: Connected to SSH!
==> azure-arm.my-example: Running local shell script: /tmp/packer-shell258158926
==> azure-arm.my-example: Provisioning step had errors: Running the cleanup provisioner, if present...
==> azure-arm.my-example:
==> azure-arm.my-example: Deleting individual resources ...
==> azure-arm.my-example: Adding to deletion queue -> Microsoft.Compute/virtualMachines : 'pkrvmgsdf8clejz'
==> azure-arm.my-example: Adding to deletion queue -> Microsoft.Network/networkInterfaces : 'pkrnigsdf8clejz'
==> azure-arm.my-example: Adding to deletion queue -> Microsoft.Network/virtualNetworks : 'pkrvngsdf8clejz'
==> azure-arm.my-example: Adding to deletion queue -> Microsoft.Network/publicIPAddresses : 'pkripgsdf8clejz'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/publicIPAddresses : 'pkripgsdf8clejz'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Compute/virtualMachines : 'pkrvmgsdf8clejz'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/virtualNetworks : 'pkrvngsdf8clejz'
==> azure-arm.my-example: Waiting for deletion of all resources...
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrnigsdf8clejz'
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkrvngsdf8clejz
==> azure-arm.my-example: Error: network.VirtualNetworksClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="InUseSubnetCannotBeDeleted" Message="Subnet pkrsngsdf8clejz is in use by /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/networkInterfaces/pkrnigsdf8clejz/ipConfigurations/ipconfig and cannot be deleted. In order to delete the subnet, delete all the resources within the subnet. See aka.ms/deletesubnet." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkrnigsdf8clejz
==> azure-arm.my-example: Error: network.InterfacesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="NicInUse" Message="Network Interface /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/networkInterfaces/pkrnigsdf8clejz is used by existing resource /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Compute/virtualMachines/pkrvmgsdf8clejz. In order to delete the network interface, it must be dissociated from the resource. To learn more, see aka.ms/deletenic." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkripgsdf8clejz
==> azure-arm.my-example: Error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/publicIPAddresses/pkripgsdf8clejz can not be deleted since it is still allocated to resource /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/networkInterfaces/pkrnigsdf8clejz/ipConfigurations/ipconfig. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/virtualNetworks : 'pkrvngsdf8clejz'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrnigsdf8clejz'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/publicIPAddresses : 'pkripgsdf8clejz'
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkrvngsdf8clejz
==> azure-arm.my-example: Error: network.VirtualNetworksClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="InUseSubnetCannotBeDeleted" Message="Subnet pkrsngsdf8clejz is in use by /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/networkInterfaces/pkrnigsdf8clejz/ipConfigurations/ipconfig and cannot be deleted. In order to delete the subnet, delete all the resources within the subnet. See aka.ms/deletesubnet." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkrnigsdf8clejz
==> azure-arm.my-example: Error: network.InterfacesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="NicInUse" Message="Network Interface /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/networkInterfaces/pkrnigsdf8clejz is used by existing resource /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Compute/virtualMachines/pkrvmgsdf8clejz. In order to delete the network interface, it must be dissociated from the resource. To learn more, see aka.ms/deletenic." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkripgsdf8clejz
==> azure-arm.my-example: Error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/publicIPAddresses/pkripgsdf8clejz can not be deleted since it is still allocated to resource /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/networkInterfaces/pkrnigsdf8clejz/ipConfigurations/ipconfig. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/virtualNetworks : 'pkrvngsdf8clejz'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/publicIPAddresses : 'pkripgsdf8clejz'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrnigsdf8clejz'
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkrvngsdf8clejz
==> azure-arm.my-example: Error: network.VirtualNetworksClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="InUseSubnetCannotBeDeleted" Message="Subnet pkrsngsdf8clejz is in use by /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/networkInterfaces/pkrnigsdf8clejz/ipConfigurations/ipconfig and cannot be deleted. In order to delete the subnet, delete all the resources within the subnet. See aka.ms/deletesubnet." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkrnigsdf8clejz
==> azure-arm.my-example: Error: network.InterfacesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="NicInUse" Message="Network Interface /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/networkInterfaces/pkrnigsdf8clejz is used by existing resource /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Compute/virtualMachines/pkrvmgsdf8clejz. In order to delete the network interface, it must be dissociated from the resource. To learn more, see aka.ms/deletenic." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkripgsdf8clejz
==> azure-arm.my-example: Error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/publicIPAddresses/pkripgsdf8clejz can not be deleted since it is still allocated to resource /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/networkInterfaces/pkrnigsdf8clejz/ipConfigurations/ipconfig. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/virtualNetworks : 'pkrvngsdf8clejz'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrnigsdf8clejz'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/publicIPAddresses : 'pkripgsdf8clejz'
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkrvngsdf8clejz
==> azure-arm.my-example: Error: network.VirtualNetworksClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="InUseSubnetCannotBeDeleted" Message="Subnet pkrsngsdf8clejz is in use by /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/networkInterfaces/pkrnigsdf8clejz/ipConfigurations/ipconfig and cannot be deleted. In order to delete the subnet, delete all the resources within the subnet. See aka.ms/deletesubnet." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkripgsdf8clejz
==> azure-arm.my-example: Error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/publicIPAddresses/pkripgsdf8clejz can not be deleted since it is still allocated to resource /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-gsdf8clejz/providers/Microsoft.Network/networkInterfaces/pkrnigsdf8clejz/ipConfigurations/ipconfig. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/virtualNetworks : 'pkrvngsdf8clejz'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/publicIPAddresses : 'pkripgsdf8clejz'
==> azure-arm.my-example:  Deleting -> image : 'https://idjvirtualmachines.blob.core.windows.net/images/pkrosgsdf8clejz.vhd'
==> azure-arm.my-example: Removing the created Deployment object: 'pkrdpgsdf8clejz'
==> azure-arm.my-example:
==> azure-arm.my-example: Cleanup requested, deleting resource group ...
==> azure-arm.my-example: Resource group has been deleted.
Build 'azure-arm.my-example' errored after 4 minutes 47 seconds: Error executing script: /tmp/packer-shell258158926

Please see output above for more information.

==> Wait completed after 4 minutes 47 seconds

==> Some builds didn't complete successfully and had errors:
--> azure-arm.my-example: Error executing script: /tmp/packer-shell258158926

Please see output above for more information.

==> Builds finished but no artifacts were created.

It seems it did not like something with our steps.

Let’s run with debug. You’ll have to press enter to step it along in this mode.

$ packer build -debug .
Debug mode enabled. Builds will not be parallelized.
azure-arm.my-example: output will be in this color.

==> azure-arm.my-example: Running builder ...
==> azure-arm.my-example: Getting tokens using client secret
==> azure-arm.my-example: Getting tokens using client secret
    azure-arm.my-example: Creating Azure Resource Manager (ARM) client ...
==> azure-arm.my-example: Warning: You are using Azure Packer Builder to create VHDs which is being deprecated, consider using Managed Images. Learn more https://www.packer.io/docs/builders/azure/arm#azure-arm-builder-specific-options
==> azure-arm.my-example: WARNING: Zone resiliency may not be supported in East US, checkout the docs at https://docs.microsoft.com/en-us/azure/availability-zones/
    azure-arm.my-example: temp admin user: 'packer'
    azure-arm.my-example: temp admin password: 'RXPOrwNZS1LY6UCGXnbVPI0aljeOjpCW'
    azure-arm.my-example: temp ssh key: my-example-pkrvmd2pns72ggr.pem
==> azure-arm.my-example: Creating resource group ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-d2pns72ggr'
==> azure-arm.my-example:  -> Location          : 'East US'
==> azure-arm.my-example:  -> Tags              :
==> azure-arm.my-example:  ->> dept : engineering
==> azure-arm.my-example: Pausing after run of step 'StepCreateResourceGroup'. Press enter to continue.
...

with tail -f /tmp/packer-build.txt we can see the underlying issue

...
2021/09/05 19:19:47 packer-provisioner-shell-local plugin: [INFO] (shell-local): Prepending inline script with #!/bin/sh -e
2021/09/05 19:19:47 ui: ==> azure-arm.my-example: Running local shell script: /tmp/packer-shell261250293
2021/09/05 19:19:47 packer-provisioner-shell-local plugin: [INFO] (shell-local): starting local command: chmod +x <no value>; PACKER_BUILDER_TYPE='azure-arm' PACKER_BUILD_NAME='my-example'  sudo -E sh '<no value>'
2021/09/05 19:19:47 packer-provisioner-shell-local plugin: [INFO] (shell-local communicator): Executing local shell command [chmod +x <no value>; PACKER_BUILDER_TYPE='azure-arm' PACKER_BUILD_NAME='my-example'  sudo -E sh '<no value>']
2021/09/05 19:19:47 [INFO] (telemetry) ending shell-local
2021/09/05 19:19:47 ui: ==> azure-arm.my-example: Provisioning step had errors: Running the cleanup provisioner, if present...2021/09/05 19:19:47 ui: ask: ==> azure-arm.my-example: Pausing before cleanup of step 'StepConnectSSH'. Press enter to continue.
2021/09/05 19:19:56 ui: ask: ==> azure-arm.my-example: Pausing before cleanup of step 'StepGetIPAddress'. Press enter to continue.
2021/09/05 19:19:58 ui: ask: ==> azure-arm.my-example: Pausing before cleanup of step 'StepDeployTemplate'. Press enter to continue.
...

the first command is missing values..


    execute_command = ["chmod +x {{ .Path }}; {{ .Vars }} sudo -E sh '{{ .Path }}'"]

The fact is i just borrowed that from an example. It’s really not any use to me. its a sudoer comand for some path and vars. We can see from the shell-local docs the difference between command and inline is just running a command in a temp file vs many.

I’ll nix that and run again:

build {
  sources = ["sources.azure-arm.my-example"]
  
  provisioner "shell-local" {
    inline           = [
      "apt-get update",
      "apt-get upgrade -y",
      "apt-get -y install ca-certificates curl apt-transport-https lsb-release gnupg",
      "curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | tee /etc/apt/trusted.gpg.d/microsoft.gpg > /dev/null",
      "AZ_REPO=$(lsb_release -cs) && echo \"deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main\" | tee /etc/apt/sources.list.d/azure-cli.list",
      "apt-get -y install azure-cli",
      "apt-get update",
      "/usr/sbin/waagent -force -deprovision+user && export HISTSIZE=0 && sync"
    ]
  }
}

Shell local ended up running these steps locally!. I managed to update my local Azure CLI, so i guess i got that going for me…

Fixing the VHD

I needed to change to “shell”, not “shell-local”. I realized that when it prompted me for a sudo password for ‘builder’ not packer. I also fought making sudo work in each command before i just moved it all to a script file.

The end result:

$ cat setup-azcli.sh
#!/bin/bash

export DEBIAN_FRONTEND=noninteractive

apt-get update
apt-get -y install ca-certificates curl apt-transport-https lsb-release gnupg
curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | tee /etc/apt/trusted.gpg.d/microsoft.gpg > /dev/null
AZ_REPO=$(lsb_release -cs) && echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" | sudo tee /etc/apt/sources.list.d/azure-cli.list
apt-get update
apt-get -y install azure-cli

$ cat ubuntu-test.pkr.hcl

source "azure-arm" "my-example" {
  client_id = "${var.client_id}"
  client_secret = "${var.client_secret}"
  
  resource_group_name = "packerdemo"
  storage_account = "idjvirtualmachines"

  tenant_id= "${var.tenant_id}"
  subscription_id = "${var.subscription_id}"

  capture_container_name = "images"
  capture_name_prefix = "packer"

  os_type = "Linux"
  image_publisher = "Canonical"
  image_offer = "UbuntuServer"
  image_sku = "18.04-LTS"

  location = "East US"
  vm_size = "Standard_DS2_v2"

  azure_tags = {
    dept = "engineering"
  }
}

build {
  sources = ["sources.azure-arm.my-example"]
  
  provisioner "shell" {
    
    execute_command = "chmod +x {{ .Path }}; {{ .Vars }} sudo {{ .Path }}"
    script          = "./setup-azcli.sh"
  }
}

# /usr/sbin/waagent -force -deprovision+user && export HISTSIZE=0 && sync
# DEBIAN_FRONTEND=noninteractive && sudo apt-get upgrade -o Dpkg::Options::=--force-confold -o Dpkg::Options::=--force-confdef -y --allow-downgrades --allow-remove-essential --allow-change-held-packages

You can see the key change was the shell and the provisioner shell step.

Testing

Now we can see it work:

builder@DESKTOP-QADGF36:~/Workspaces/PackerBlog$ packer build .
azure-arm.my-example: output will be in this color.

==> azure-arm.my-example: Running builder ...
==> azure-arm.my-example: Getting tokens using client secret
==> azure-arm.my-example: Getting tokens using client secret
    azure-arm.my-example: Creating Azure Resource Manager (ARM) client ...
==> azure-arm.my-example: Warning: You are using Azure Packer Builder to create VHDs which is being deprecated, consider using Managed Images. Learn more https://www.packer.io/docs/builders/azure/arm#azure-arm-builder-specific-options
==> azure-arm.my-example: WARNING: Zone resiliency may not be supported in East US, checkout the docs at https://docs.microsoft.com/en-us/azure/availability-zones/
==> azure-arm.my-example: Creating resource group ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-iiemkalhss'
==> azure-arm.my-example:  -> Location          : 'East US'
==> azure-arm.my-example:  -> Tags              :
==> azure-arm.my-example:  ->> dept : engineering
==> azure-arm.my-example: Validating deployment template ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-iiemkalhss'
==> azure-arm.my-example:  -> DeploymentName    : 'pkrdpiiemkalhss'
==> azure-arm.my-example: Deploying deployment template ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-iiemkalhss'
==> azure-arm.my-example:  -> DeploymentName    : 'pkrdpiiemkalhss'
==> azure-arm.my-example:
==> azure-arm.my-example: Getting the VM's IP address ...
==> azure-arm.my-example:  -> ResourceGroupName   : 'pkr-Resource-Group-iiemkalhss'
==> azure-arm.my-example:  -> PublicIPAddressName : 'pkripiiemkalhss'
==> azure-arm.my-example:  -> NicName             : 'pkrniiiemkalhss'
==> azure-arm.my-example:  -> Network Connection  : 'PublicEndpoint'
==> azure-arm.my-example:  -> IP Address          : '40.121.149.243'
==> azure-arm.my-example: Waiting for SSH to become available...
==> azure-arm.my-example: Connected to SSH!
==> azure-arm.my-example: Provisioning with shell script: ./setup-azcli.sh
    azure-arm.my-example: Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
    azure-arm.my-example: Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease
    azure-arm.my-example: Get:3 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1846 kB]
    azure-arm.my-example: Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
    azure-arm.my-example: Get:5 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [338 kB]
    azure-arm.my-example: Get:6 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [419 kB]
    azure-arm.my-example: Get:7 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [56.1 kB]
    azure-arm.my-example: Get:8 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1137 kB]
    azure-arm.my-example: Get:9 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [259 kB]
    azure-arm.my-example: Get:10 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [20.9 kB]
    azure-arm.my-example: Get:11 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [4732 B]
    azure-arm.my-example: Get:12 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
    azure-arm.my-example: Get:13 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [8570 kB]
    azure-arm.my-example: Get:14 http://archive.ubuntu.com/ubuntu bionic/universe Translation-en [4941 kB]
    azure-arm.my-example: Get:15 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [151 kB]
    azure-arm.my-example: Get:16 http://archive.ubuntu.com/ubuntu bionic/multiverse Translation-en [108 kB]
    azure-arm.my-example: Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [2192 kB]
    azure-arm.my-example: Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [430 kB]
    azure-arm.my-example: Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [443 kB]
    azure-arm.my-example: Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [59.9 kB]
    azure-arm.my-example: Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1748 kB]
    azure-arm.my-example: Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [375 kB]
    azure-arm.my-example: Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [27.3 kB]
    azure-arm.my-example: Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6808 B]
    azure-arm.my-example: Get:25 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [10.0 kB]
    azure-arm.my-example: Get:26 http://archive.ubuntu.com/ubuntu bionic-backports/main Translation-en [4764 B]
    azure-arm.my-example: Get:27 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [10.3 kB]
    azure-arm.my-example: Get:28 http://archive.ubuntu.com/ubuntu bionic-backports/universe Translation-en [4588 B]
    azure-arm.my-example: Fetched 23.4 MB in 5s (4859 kB/s)
    azure-arm.my-example: Reading package lists...
    azure-arm.my-example: Reading package lists...
    azure-arm.my-example: Building dependency tree...
    azure-arm.my-example: Reading state information...
    azure-arm.my-example: ca-certificates is already the newest version (20210119~18.04.1).
    azure-arm.my-example: ca-certificates set to manually installed.
    azure-arm.my-example: gnupg is already the newest version (2.2.4-1ubuntu1.4).
    azure-arm.my-example: gnupg set to manually installed.
    azure-arm.my-example: lsb-release is already the newest version (9.20170808ubuntu1).
    azure-arm.my-example: lsb-release set to manually installed.
    azure-arm.my-example: The following package was automatically installed and is no longer required:
    azure-arm.my-example:   linux-headers-4.15.0-151
    azure-arm.my-example: Use 'sudo apt autoremove' to remove it.
    azure-arm.my-example: The following additional packages will be installed:
    azure-arm.my-example:   libcurl4
    azure-arm.my-example: The following NEW packages will be installed:
    azure-arm.my-example:   apt-transport-https
    azure-arm.my-example: The following packages will be upgraded:
    azure-arm.my-example:   curl libcurl4
    azure-arm.my-example: 2 upgraded, 1 newly installed, 0 to remove and 25 not upgraded.
    azure-arm.my-example: Need to get 379 kB of archives.
    azure-arm.my-example: After this operation, 153 kB of additional disk space will be used.
    azure-arm.my-example: Get:1 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 apt-transport-https all 1.6.12ubuntu0.2 [1696 B]
    azure-arm.my-example: Get:2 http://security.ubuntu.com/ubuntu bionic-security/main amd64 curl amd64 7.58.0-2ubuntu3.14 [159 kB]
    azure-arm.my-example: Get:3 http://security.ubuntu.com/ubuntu bionic-security/main amd64 libcurl4 amd64 7.58.0-2ubuntu3.14 [219 kB]
    azure-arm.my-example: Fetched 379 kB in 0s (3822 kB/s)
    azure-arm.my-example: Selecting previously unselected package apt-transport-https.
    azure-arm.my-example: (Reading database ... 76951 files and directories currently installed.)
    azure-arm.my-example: Preparing to unpack .../apt-transport-https_1.6.12ubuntu0.2_all.deb ...
    azure-arm.my-example: Unpacking apt-transport-https (1.6.12ubuntu0.2) ...
    azure-arm.my-example: Preparing to unpack .../curl_7.58.0-2ubuntu3.14_amd64.deb ...
    azure-arm.my-example: Unpacking curl (7.58.0-2ubuntu3.14) over (7.58.0-2ubuntu3.13) ...
    azure-arm.my-example: Preparing to unpack .../libcurl4_7.58.0-2ubuntu3.14_amd64.deb ...
    azure-arm.my-example: Unpacking libcurl4:amd64 (7.58.0-2ubuntu3.14) over (7.58.0-2ubuntu3.13) ...
    azure-arm.my-example: Setting up apt-transport-https (1.6.12ubuntu0.2) ...
    azure-arm.my-example: Setting up libcurl4:amd64 (7.58.0-2ubuntu3.14) ...
    azure-arm.my-example: Setting up curl (7.58.0-2ubuntu3.14) ...
    azure-arm.my-example: Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
    azure-arm.my-example: Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
==> azure-arm.my-example: gpg: WARNING: unsafe ownership on homedir '/home/packer/.gnupg'
    azure-arm.my-example: deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ bionic main
    azure-arm.my-example: Get:1 http://azure.archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
    azure-arm.my-example: Get:2 http://azure.archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
    azure-arm.my-example: Get:3 http://azure.archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
    azure-arm.my-example: Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease
    azure-arm.my-example: Get:5 https://packages.microsoft.com/repos/azure-cli bionic InRelease [3965 B]
    azure-arm.my-example: Get:6 http://azure.archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1019 kB]
    azure-arm.my-example: Get:7 http://azure.archive.ubuntu.com/ubuntu bionic/main Translation-en [516 kB]
    azure-arm.my-example: Get:8 http://azure.archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [9184 B]
    azure-arm.my-example: Get:9 http://azure.archive.ubuntu.com/ubuntu bionic/restricted Translation-en [3584 B]
    azure-arm.my-example: Get:10 http://azure.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [8570 kB]
    azure-arm.my-example: Get:11 http://azure.archive.ubuntu.com/ubuntu bionic/universe Translation-en [4941 kB]
    azure-arm.my-example: Get:12 http://azure.archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [151 kB]
    azure-arm.my-example: Get:13 http://azure.archive.ubuntu.com/ubuntu bionic/multiverse Translation-en [108 kB]
    azure-arm.my-example: Get:14 http://azure.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [2192 kB]
    azure-arm.my-example: Get:15 http://azure.archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [430 kB]
    azure-arm.my-example: Get:16 http://azure.archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [443 kB]
    azure-arm.my-example: Get:17 http://azure.archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [59.9 kB]
    azure-arm.my-example: Get:18 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1748 kB]
    azure-arm.my-example: Get:19 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [375 kB]
    azure-arm.my-example: Get:20 http://azure.archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [27.3 kB]
    azure-arm.my-example: Get:21 http://azure.archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6808 B]
    azure-arm.my-example: Get:22 http://azure.archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [10.0 kB]
    azure-arm.my-example: Get:23 http://azure.archive.ubuntu.com/ubuntu bionic-backports/main Translation-en [4764 B]
    azure-arm.my-example: Get:24 http://azure.archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [10.3 kB]
    azure-arm.my-example: Get:25 http://azure.archive.ubuntu.com/ubuntu bionic-backports/universe Translation-en [4588 B]
    azure-arm.my-example: Get:26 https://packages.microsoft.com/repos/azure-cli bionic/main amd64 Packages [15.0 kB]
    azure-arm.my-example: Fetched 21.1 MB in 4s (5775 kB/s)
    azure-arm.my-example: Reading package lists...
    azure-arm.my-example: Reading package lists...
    azure-arm.my-example: Building dependency tree...
    azure-arm.my-example: Reading state information...
    azure-arm.my-example: The following package was automatically installed and is no longer required:
    azure-arm.my-example:   linux-headers-4.15.0-151
    azure-arm.my-example: Use 'sudo apt autoremove' to remove it.
    azure-arm.my-example: The following NEW packages will be installed:
    azure-arm.my-example:   azure-cli
    azure-arm.my-example: 0 upgraded, 1 newly installed, 0 to remove and 29 not upgraded.
    azure-arm.my-example: Need to get 63.1 MB of archives.
    azure-arm.my-example: After this operation, 952 MB of additional disk space will be used.
    azure-arm.my-example: Get:1 https://packages.microsoft.com/repos/azure-cli bionic/main amd64 azure-cli all 2.27.2-1~bionic [63.1 MB]
    azure-arm.my-example: Fetched 63.1 MB in 1s (51.3 MB/s)
    azure-arm.my-example: Selecting previously unselected package azure-cli.
    azure-arm.my-example: (Reading database ... 76955 files and directories currently installed.)
    azure-arm.my-example: Preparing to unpack .../azure-cli_2.27.2-1~bionic_all.deb ...
    azure-arm.my-example: Unpacking azure-cli (2.27.2-1~bionic) ...
    azure-arm.my-example: Setting up azure-cli (2.27.2-1~bionic) ...
==> azure-arm.my-example: Querying the machine's properties ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-iiemkalhss'
==> azure-arm.my-example:  -> ComputeName       : 'pkrvmiiemkalhss'
==> azure-arm.my-example:  -> OS Disk           : 'https://idjvirtualmachines.blob.core.windows.net/images/pkrosiiemkalhss.vhd'
==> azure-arm.my-example: Querying the machine's additional disks properties ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-iiemkalhss'
==> azure-arm.my-example:  -> ComputeName       : 'pkrvmiiemkalhss'
==> azure-arm.my-example: Powering off machine ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-iiemkalhss'
==> azure-arm.my-example:  -> ComputeName       : 'pkrvmiiemkalhss'
==> azure-arm.my-example: Capturing image ...
==> azure-arm.my-example:  -> Compute ResourceGroupName : 'pkr-Resource-Group-iiemkalhss'
==> azure-arm.my-example:  -> Compute Name              : 'pkrvmiiemkalhss'
==> azure-arm.my-example:  -> Compute Location          : 'East US'
==> azure-arm.my-example:
==> azure-arm.my-example: Deleting individual resources ...
==> azure-arm.my-example: Adding to deletion queue -> Microsoft.Compute/virtualMachines : 'pkrvmiiemkalhss'
==> azure-arm.my-example: Adding to deletion queue -> Microsoft.Network/networkInterfaces : 'pkrniiiemkalhss'
==> azure-arm.my-example: Adding to deletion queue -> Microsoft.Network/virtualNetworks : 'pkrvniiemkalhss'
==> azure-arm.my-example: Adding to deletion queue -> Microsoft.Network/publicIPAddresses : 'pkripiiemkalhss'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrniiiemkalhss'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/virtualNetworks : 'pkrvniiemkalhss'
==> azure-arm.my-example: Waiting for deletion of all resources...
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/publicIPAddresses : 'pkripiiemkalhss'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Compute/virtualMachines : 'pkrvmiiemkalhss'
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkrniiiemkalhss
==> azure-arm.my-example: Error: network.InterfacesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="NicInUse" Message="Network Interface /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-iiemkalhss/providers/Microsoft.Network/networkInterfaces/pkrniiiemkalhss is used by existing resource /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-iiemkalhss/providers/Microsoft.Compute/virtualMachines/pkrvmiiemkalhss. In order to delete the network interface, it must be dissociated from the resource. To learn more, see aka.ms/deletenic." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkripiiemkalhss
==> azure-arm.my-example: Error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-iiemkalhss/providers/Microsoft.Network/publicIPAddresses/pkripiiemkalhss can not be deleted since it is still allocated to resource /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-iiemkalhss/providers/Microsoft.Network/networkInterfaces/pkrniiiemkalhss/ipConfigurations/ipconfig. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkrvniiemkalhss
==> azure-arm.my-example: Error: network.VirtualNetworksClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="InUseSubnetCannotBeDeleted" Message="Subnet pkrsniiemkalhss is in use by /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-iiemkalhss/providers/Microsoft.Network/networkInterfaces/pkrniiiemkalhss/ipConfigurations/ipconfig and cannot be deleted. In order to delete the subnet, delete all the resources within the subnet. See aka.ms/deletesubnet." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrniiiemkalhss'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/publicIPAddresses : 'pkripiiemkalhss'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/virtualNetworks : 'pkrvniiemkalhss'
==> azure-arm.my-example:  Deleting -> image : 'https://idjvirtualmachines.blob.core.windows.net/images/pkrosiiemkalhss.vhd'
==> azure-arm.my-example: Removing the created Deployment object: 'pkrdpiiemkalhss'
==> azure-arm.my-example:
==> azure-arm.my-example: Cleanup requested, deleting resource group ...
==> azure-arm.my-example: Resource group has been deleted.
Build 'azure-arm.my-example' finished after 5 minutes 36 seconds.

==> Wait completed after 5 minutes 36 seconds

==> Builds finished. The artifacts of successful builds are:
--> azure-arm.my-example: Azure.ResourceManagement.VMImage:

OSType: Linux
StorageAccountLocation: eastus
OSDiskUri: https://idjvirtualmachines.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-osDisk.7738a285-8975-4d52-8d87-e318b5eca0cd.vhd
OSDiskUriReadOnlySas: https://idjvirtualmachines.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-osDisk.7738a285-8975-4d52-8d87-e318b5eca0cd.vhd?se=2021-10-06T03%3A30%3A04Z&sig=bAdnqHz0IQBpHwCU7Acjfspyt5N0QgRFIKQnqVDbghw%3D&sp=r&sr=b&sv=2018-03-28
TemplateUri: https://idjvirtualmachines.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-vmTemplate.7738a285-8975-4d52-8d87-e318b5eca0cd.json
TemplateUriReadOnlySas: https://idjvirtualmachines.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-vmTemplate.7738a285-8975-4d52-8d87-e318b5eca0cd.json?se=2021-10-06T03%3A30%3A04Z&sig=CsNgkDqxjWtOwit%2BUqQC1mzBrfvUs0RLv5c7%2B5RzKlA%3D&sp=r&sr=b&sv=2018-03-28

And we can now see the images in Azure

/content/images/2021/09/packer-build-01.png

Now on it’s face, just having that image in and of itself is not that expensive. We would expect to pay $1.42/mo just for the 30Gb page blob in the hot tier on that account.

/content/images/2021/09/packer-build-02.png

Creating a VM from disk

First, we need to make a managed disk from the VHD

$ az disk create -g packerdemo -n myDiskName --sku Premium_LRS --location eastus --size-gb 30 --source https://idjvirtualmachines.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-osDisk.7738a285-8975-4d52-8d87-e318b5eca0cd.vhd
{
  "burstingEnabled": null,
  "creationData": {
    "createOption": "Import",
    "galleryImageReference": null,
    "imageReference": null,
    "logicalSectorSize": null,
    "sourceResourceId": null,
    "sourceUniqueId": null,
    "sourceUri": "https://idjvirtualmachines.blob.core.windows.net/system/Microsoft.Compute/Images/images/packer-osDisk.7738a285-8975-4d52-8d87-e318b5eca0cd.vhd",
    "storageAccountId": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Storage/storageAccounts/idjvirtualmachines",
    "uploadSizeBytes": null
  },
  "diskAccessId": null,
  "diskIopsReadOnly": null,
  "diskIopsReadWrite": 120,
  "diskMBpsReadOnly": null,
  "diskMBpsReadWrite": 25,
  "diskSizeBytes": 32213303296,
  "diskSizeGb": 30,
  "diskState": "Unattached",
  "encryption": {
    "diskEncryptionSetId": null,
    "type": "EncryptionAtRestWithPlatformKey"
  },
  "encryptionSettingsCollection": null,
  "extendedLocation": null,
  "hyperVGeneration": "V1",
  "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Compute/disks/myDiskName",
  "location": "eastus",
  "managedBy": null,
  "managedByExtended": null,
  "maxShares": null,
  "name": "myDiskName",
  "networkAccessPolicy": "AllowAll",
  "osType": null,
  "propertyUpdatesInProgress": null,
  "provisioningState": "Succeeded",
  "purchasePlan": null,
  "resourceGroup": "packerdemo",
  "securityProfile": null,
  "shareInfo": null,
  "sku": {
    "name": "Premium_LRS",
    "tier": "Premium"
  },
  "supportsHibernation": null,
  "tags": {},
  "tier": "P4",
  "timeCreated": "2021-09-06T03:52:27.803836+00:00",
  "type": "Microsoft.Compute/disks",
  "uniqueId": "409cd503-929f-4c8d-98e5-78e4793539a5",
  "zones": null
}

Then we can make a VM that uses that managed disk just as easily

$ az vm create -g packerdemo --location eastus --name myLinuxVM --os-type linux --attach-os-disk myDiskName
It is recommended to use parameter "--public-ip-sku Standard" to create new VM with Standard public IP. Please note that the default public IP used for VM creation will be changed from Basic to Standard in the future.
{
  "fqdns": "",
  "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Compute/virtualMachines/myLinuxVM",
  "location": "eastus",
  "macAddress": "00-0D-3A-98-EE-A0",
  "powerState": "VM running",
  "privateIpAddress": "10.0.0.4",
  "publicIpAddress": "13.68.132.179",
  "resourceGroup": "packerdemo",
  "zones": ""
}

We see the results in our resource group:

/content/images/2021/09/packer-build-03.png

However, despite what i can try, i cannot seem to SSH in

builder@DESKTOP-QADGF36:~/Workspaces/PackerBlog$ chmod 400 my-example-pkrvmd2pns72ggr.pem
builder@DESKTOP-QADGF36:~/Workspaces/PackerBlog$ mv my-example-pkrvmd2pns72ggr.pem ~/.ssh
builder@DESKTOP-QADGF36:~/Workspaces/PackerBlog$ ssh -i ~/.ssh/my-example-pkrvmd2pns72ggr.pem packer@13.68.132.179
packer@13.68.132.179: Permission denied (publickey).
builder@DESKTOP-QADGF36:~/Workspaces/PackerBlog$ ssh -i ~/.ssh/my-example-pkrvmd2pns72ggr.pem ubuntu@13.68.132.179
Connection closed by 13.68.132.179 port 22
builder@DESKTOP-QADGF36:~/Workspaces/PackerBlog$ ssh -i ~/.ssh/my-example-pkrvmd2pns72ggr.pem root@13.68.132.179
root@13.68.132.179: Permission denied (publickey).

However, this was easily remidied by using the password reset in Azure:

/content/images/2021/09/packer-build-04.png

and we can SSH in and verify that indeed the Azure CLI was installed on this base Ubuntu image

uilder@DESKTOP-QADGF36:~/Workspaces/PackerBlog$ ssh ubuntu@13.68.132.179
ubuntu@13.68.132.179's password:
Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 5.4.0-1055-azure x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Mon Sep  6 04:07:36 UTC 2021

  System load:  0.0               Processes:           109
  Usage of /:   8.8% of 28.90GB   Users logged in:     0
  Memory usage: 5%                IP address for eth0: 10.0.0.4
  Swap usage:   0%


38 updates can be applied immediately.
34 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable

New release '20.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.



The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

ubuntu@pkrvmiiemkalhss:~$ az version
{
  "azure-cli": "2.27.2",
  "azure-cli-core": "2.27.2",
  "azure-cli-telemetry": "1.0.6",
  "extensions": {}
}

Cleanup

First we need to remove the VM:

$ az vm delete -g packerdemo -n myLinuxVM --subscription "Pay-As-You-Go"
Are you sure you want to perform this operation? (y/n): y

Then we will want to delete the managed disk

$ az disk delete --name myDiskName -g packerdemo
Are you sure you want to perform this operation? (y/n): y

Using Managed Images

Managed images, instead of VHDs, are the preferred way now (though VHDs are very exportable).

We just need to change the lines in our builder’s file:

  # VHD style
  # resource_group_name = "packerdemo"
  # storage_account = "idjvirtualmachines"
  # capture_container_name = "images"
  # capture_name_prefix = "packer"

  # Managed Image style
  managed_image_resource_group_name = "packerdemo"
  managed_image_name = "myPackerImage"

In full context:

source "azure-arm" "my-example" {
  client_id = "${var.client_id}"
  client_secret = "${var.client_secret}"
  

  tenant_id= "${var.tenant_id}"
  subscription_id = "${var.subscription_id}"

  # VHD style
  # resource_group_name = "packerdemo"
  # storage_account = "idjvirtualmachines"
  # capture_container_name = "images"
  # capture_name_prefix = "packer"

  # Managed Image style
  managed_image_resource_group_name = "packerdemo"
  managed_image_name = "myPackerImage"

  os_type = "Linux"
  image_publisher = "Canonical"
  image_offer = "UbuntuServer"
  image_sku = "18.04-LTS"

  location = "East US"
  vm_size = "Standard_DS2_v2"

  azure_tags = {
    dept = "engineering"
  }
}

build {
  sources = ["sources.azure-arm.my-example"]
  
  provisioner "shell" {
    
    execute_command = "chmod +x {{ .Path }}; {{ .Vars }} sudo {{ .Path }}"
    script          = "./setup-azcli.sh"
  }
}

Now when we build, we should get a managed image that we can use in VMSS and launching VMs.

$ packer build .
azure-arm.my-example: output will be in this color.

==> azure-arm.my-example: Running builder ...
==> azure-arm.my-example: Getting tokens using client secret
==> azure-arm.my-example: Getting tokens using client secret
    azure-arm.my-example: Creating Azure Resource Manager (ARM) client ...
==> azure-arm.my-example: WARNING: Zone resiliency may not be supported in East US, checkout the docs at https://docs.microsoft.com/en-us/azure/availability-zones/
==> azure-arm.my-example: Creating resource group ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-u49drt5w46'
==> azure-arm.my-example:  -> Location          : 'East US'
==> azure-arm.my-example:  -> Tags              :
==> azure-arm.my-example:  ->> dept : engineering
==> azure-arm.my-example: Validating deployment template ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-u49drt5w46'
==> azure-arm.my-example:  -> DeploymentName    : 'pkrdpu49drt5w46'
==> azure-arm.my-example: Deploying deployment template ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-u49drt5w46'
==> azure-arm.my-example:  -> DeploymentName    : 'pkrdpu49drt5w46'
==> azure-arm.my-example:
==> azure-arm.my-example: Getting the VM's IP address ...
==> azure-arm.my-example:  -> ResourceGroupName   : 'pkr-Resource-Group-u49drt5w46'
==> azure-arm.my-example:  -> PublicIPAddressName : 'pkripu49drt5w46'
==> azure-arm.my-example:  -> NicName             : 'pkrniu49drt5w46'
==> azure-arm.my-example:  -> Network Connection  : 'PublicEndpoint'
==> azure-arm.my-example:  -> IP Address          : '20.102.67.207'
==> azure-arm.my-example: Waiting for SSH to become available...
==> azure-arm.my-example: Connected to SSH!
==> azure-arm.my-example: Provisioning with shell script: ./setup-azcli.sh
    azure-arm.my-example: Hit:1 http://azure.archive.ubuntu.com/ubuntu bionic InRelease
    azure-arm.my-example: Get:2 http://azure.archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
    azure-arm.my-example: Get:3 http://azure.archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
    azure-arm.my-example: Get:4 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
    azure-arm.my-example: Get:5 http://azure.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [8570 kB]
    azure-arm.my-example: Get:6 http://azure.archive.ubuntu.com/ubuntu bionic/universe Translation-en [4941 kB]
    azure-arm.my-example: Get:7 http://azure.archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [151 kB]
    azure-arm.my-example: Get:8 http://azure.archive.ubuntu.com/ubuntu bionic/multiverse Translation-en [108 kB]
    azure-arm.my-example: Get:9 http://azure.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [2192 kB]
    azure-arm.my-example: Get:10 http://azure.archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [430 kB]
    azure-arm.my-example: Get:11 http://azure.archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [443 kB]
    azure-arm.my-example: Get:12 http://azure.archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [59.9 kB]
    azure-arm.my-example: Get:13 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1748 kB]
    azure-arm.my-example: Get:14 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [375 kB]
    azure-arm.my-example: Get:15 http://azure.archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [27.3 kB]
    azure-arm.my-example: Get:16 http://azure.archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6808 B]
    azure-arm.my-example: Get:17 http://azure.archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [10.0 kB]
    azure-arm.my-example: Get:18 http://azure.archive.ubuntu.com/ubuntu bionic-backports/main Translation-en [4764 B]
    azure-arm.my-example: Get:19 http://azure.archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [10.3 kB]
    azure-arm.my-example: Get:20 http://azure.archive.ubuntu.com/ubuntu bionic-backports/universe Translation-en [4588 B]
    azure-arm.my-example: Get:21 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1846 kB]
    azure-arm.my-example: Get:22 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [338 kB]
    azure-arm.my-example: Get:23 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [419 kB]
    azure-arm.my-example: Get:24 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [56.1 kB]
    azure-arm.my-example: Get:25 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1137 kB]
    azure-arm.my-example: Get:26 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [259 kB]
    azure-arm.my-example: Get:27 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [20.9 kB]
    azure-arm.my-example: Get:28 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [4732 B]
    azure-arm.my-example: Fetched 23.4 MB in 4s (5564 kB/s)
    azure-arm.my-example: Reading package lists...
    azure-arm.my-example: Reading package lists...
    azure-arm.my-example: Building dependency tree...
    azure-arm.my-example: Reading state information...
    azure-arm.my-example: lsb-release is already the newest version (9.20170808ubuntu1).
    azure-arm.my-example: lsb-release set to manually installed.
    azure-arm.my-example: ca-certificates is already the newest version (20210119~18.04.1).
    azure-arm.my-example: ca-certificates set to manually installed.
    azure-arm.my-example: gnupg is already the newest version (2.2.4-1ubuntu1.4).
    azure-arm.my-example: gnupg set to manually installed.
    azure-arm.my-example: The following package was automatically installed and is no longer required:
    azure-arm.my-example:   linux-headers-4.15.0-151
    azure-arm.my-example: Use 'sudo apt autoremove' to remove it.
    azure-arm.my-example: The following additional packages will be installed:
    azure-arm.my-example:   libcurl4
    azure-arm.my-example: The following NEW packages will be installed:
    azure-arm.my-example:   apt-transport-https
    azure-arm.my-example: The following packages will be upgraded:
    azure-arm.my-example:   curl libcurl4
    azure-arm.my-example: 2 upgraded, 1 newly installed, 0 to remove and 28 not upgraded.
    azure-arm.my-example: Need to get 382 kB of archives.
    azure-arm.my-example: After this operation, 154 kB of additional disk space will be used.
    azure-arm.my-example: Get:1 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 apt-transport-https all 1.6.14 [4348 B]
    azure-arm.my-example: Get:2 http://azure.archive.ubuntu.com/ubuntu bionic-updates/main amd64 curl amd64 7.58.0-2ubuntu3.14 [159 kB]
    azure-arm.my-example: Get:3 http://azure.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libcurl4 amd64 7.58.0-2ubuntu3.14 [219 kB]
    azure-arm.my-example: Fetched 382 kB in 0s (12.4 MB/s)
    azure-arm.my-example: Selecting previously unselected package apt-transport-https.
    azure-arm.my-example: (Reading database ... 76951 files and directories currently installed.)
    azure-arm.my-example: Preparing to unpack .../apt-transport-https_1.6.14_all.deb ...
    azure-arm.my-example: Unpacking apt-transport-https (1.6.14) ...
    azure-arm.my-example: Preparing to unpack .../curl_7.58.0-2ubuntu3.14_amd64.deb ...
    azure-arm.my-example: Unpacking curl (7.58.0-2ubuntu3.14) over (7.58.0-2ubuntu3.13) ...
    azure-arm.my-example: Preparing to unpack .../libcurl4_7.58.0-2ubuntu3.14_amd64.deb ...
    azure-arm.my-example: Unpacking libcurl4:amd64 (7.58.0-2ubuntu3.14) over (7.58.0-2ubuntu3.13) ...
    azure-arm.my-example: Setting up apt-transport-https (1.6.14) ...
    azure-arm.my-example: Setting up libcurl4:amd64 (7.58.0-2ubuntu3.14) ...
    azure-arm.my-example: Setting up curl (7.58.0-2ubuntu3.14) ...
    azure-arm.my-example: Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
    azure-arm.my-example: Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
==> azure-arm.my-example: gpg: WARNING: unsafe ownership on homedir '/home/packer/.gnupg'
    azure-arm.my-example: deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ bionic main
    azure-arm.my-example: Hit:1 http://azure.archive.ubuntu.com/ubuntu bionic InRelease
    azure-arm.my-example: Hit:2 http://azure.archive.ubuntu.com/ubuntu bionic-updates InRelease
    azure-arm.my-example: Hit:3 http://azure.archive.ubuntu.com/ubuntu bionic-backports InRelease
    azure-arm.my-example: Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease
    azure-arm.my-example: Get:5 https://packages.microsoft.com/repos/azure-cli bionic InRelease [3965 B]
    azure-arm.my-example: Get:6 https://packages.microsoft.com/repos/azure-cli bionic/main amd64 Packages [15.0 kB]
    azure-arm.my-example: Fetched 19.0 kB in 0s (44.8 kB/s)
    azure-arm.my-example: Reading package lists...
    azure-arm.my-example: Reading package lists...
    azure-arm.my-example: Building dependency tree...
    azure-arm.my-example: Reading state information...
    azure-arm.my-example: The following package was automatically installed and is no longer required:
    azure-arm.my-example:   linux-headers-4.15.0-151
    azure-arm.my-example: Use 'sudo apt autoremove' to remove it.
    azure-arm.my-example: The following NEW packages will be installed:
    azure-arm.my-example:   azure-cli
    azure-arm.my-example: 0 upgraded, 1 newly installed, 0 to remove and 28 not upgraded.
    azure-arm.my-example: Need to get 63.1 MB of archives.
    azure-arm.my-example: After this operation, 952 MB of additional disk space will be used.
    azure-arm.my-example: Get:1 https://packages.microsoft.com/repos/azure-cli bionic/main amd64 azure-cli all 2.27.2-1~bionic [63.1 MB]
    azure-arm.my-example: Fetched 63.1 MB in 1s (56.8 MB/s)
    azure-arm.my-example: Selecting previously unselected package azure-cli.
    azure-arm.my-example: (Reading database ... 76955 files and directories currently installed.)
    azure-arm.my-example: Preparing to unpack .../azure-cli_2.27.2-1~bionic_all.deb ...
    azure-arm.my-example: Unpacking azure-cli (2.27.2-1~bionic) ...
    azure-arm.my-example: Setting up azure-cli (2.27.2-1~bionic) ...
==> azure-arm.my-example: Querying the machine's properties ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-u49drt5w46'
==> azure-arm.my-example:  -> ComputeName       : 'pkrvmu49drt5w46'
==> azure-arm.my-example:  -> Managed OS Disk   : '/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-u49drt5w46/providers/Microsoft.Compute/disks/pkrosu49drt5w46'
==> azure-arm.my-example: Querying the machine's additional disks properties ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-u49drt5w46'
==> azure-arm.my-example:  -> ComputeName       : 'pkrvmu49drt5w46'
==> azure-arm.my-example: Powering off machine ...
==> azure-arm.my-example:  -> ResourceGroupName : 'pkr-Resource-Group-u49drt5w46'
==> azure-arm.my-example:  -> ComputeName       : 'pkrvmu49drt5w46'
==> azure-arm.my-example: Capturing image ...
==> azure-arm.my-example:  -> Compute ResourceGroupName : 'pkr-Resource-Group-u49drt5w46'
==> azure-arm.my-example:  -> Compute Name              : 'pkrvmu49drt5w46'
==> azure-arm.my-example:  -> Compute Location          : 'East US'
==> azure-arm.my-example:  -> Image ResourceGroupName   : 'packerdemo'
==> azure-arm.my-example:  -> Image Name                : 'myPackerImage'
==> azure-arm.my-example:  -> Image Location            : 'East US'
==> azure-arm.my-example:
==> azure-arm.my-example: Deleting individual resources ...
==> azure-arm.my-example: Adding to deletion queue -> Microsoft.Compute/virtualMachines : 'pkrvmu49drt5w46'
==> azure-arm.my-example: Adding to deletion queue -> Microsoft.Network/networkInterfaces : 'pkrniu49drt5w46'
==> azure-arm.my-example: Adding to deletion queue -> Microsoft.Network/virtualNetworks : 'pkrvnu49drt5w46'
==> azure-arm.my-example: Adding to deletion queue -> Microsoft.Network/publicIPAddresses : 'pkripu49drt5w46'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrniu49drt5w46'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/virtualNetworks : 'pkrvnu49drt5w46'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/publicIPAddresses : 'pkripu49drt5w46'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Compute/virtualMachines : 'pkrvmu49drt5w46'
==> azure-arm.my-example: Waiting for deletion of all resources...
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkrniu49drt5w46
==> azure-arm.my-example: Error: network.InterfacesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="NicInUse" Message="Network Interface /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-u49drt5w46/providers/Microsoft.Network/networkInterfaces/pkrniu49drt5w46 is used by existing resource /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-u49drt5w46/providers/Microsoft.Compute/virtualMachines/pkrvmu49drt5w46. In order to delete the network interface, it must be dissociated from the resource. To learn more, see aka.ms/deletenic." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkripu49drt5w46
==> azure-arm.my-example: Error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-u49drt5w46/providers/Microsoft.Network/publicIPAddresses/pkripu49drt5w46 can not be deleted since it is still allocated to resource /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-u49drt5w46/providers/Microsoft.Network/networkInterfaces/pkrniu49drt5w46/ipConfigurations/ipconfig. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkrvnu49drt5w46
==> azure-arm.my-example: Error: network.VirtualNetworksClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="InUseSubnetCannotBeDeleted" Message="Subnet pkrsnu49drt5w46 is in use by /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-u49drt5w46/providers/Microsoft.Network/networkInterfaces/pkrniu49drt5w46/ipConfigurations/ipconfig and cannot be deleted. In order to delete the subnet, delete all the resources within the subnet. See aka.ms/deletesubnet." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrniu49drt5w46'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/publicIPAddresses : 'pkripu49drt5w46'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/virtualNetworks : 'pkrvnu49drt5w46'
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkripu49drt5w46
==> azure-arm.my-example: Error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-u49drt5w46/providers/Microsoft.Network/publicIPAddresses/pkripu49drt5w46 can not be deleted since it is still allocated to resource /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-u49drt5w46/providers/Microsoft.Network/networkInterfaces/pkrniu49drt5w46/ipConfigurations/ipconfig. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Error deleting resource. Will retry.
==> azure-arm.my-example: Name: pkrvnu49drt5w46
==> azure-arm.my-example: Error: network.VirtualNetworksClient#Delete: Failure sending request: StatusCode=0 -- Original Error: Code="InUseSubnetCannotBeDeleted" Message="Subnet pkrsnu49drt5w46 is in use by /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-u49drt5w46/providers/Microsoft.Network/networkInterfaces/pkrniu49drt5w46/ipConfigurations/ipconfig and cannot be deleted. In order to delete the subnet, delete all the resources within the subnet. See aka.ms/deletesubnet." Details=[]
==> azure-arm.my-example:
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/publicIPAddresses : 'pkripu49drt5w46'
==> azure-arm.my-example: Attempting deletion -> Microsoft.Network/virtualNetworks : 'pkrvnu49drt5w46'
==> azure-arm.my-example:  Deleting -> Microsoft.Compute/disks : '/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-u49drt5w46/providers/Microsoft.Compute/disks/pkrosu49drt5w46'
==> azure-arm.my-example: Removing the created Deployment object: 'pkrdpu49drt5w46'
==> azure-arm.my-example:
==> azure-arm.my-example: Cleanup requested, deleting resource group ...
==> azure-arm.my-example: Resource group has been deleted.
Build 'azure-arm.my-example' finished after 7 minutes 1 second.

==> Wait completed after 7 minutes 1 second

==> Builds finished. The artifacts of successful builds are:
--> azure-arm.my-example: Azure.ResourceManagement.VMImage:

OSType: Linux
ManagedImageResourceGroupName: packerdemo
ManagedImageName: myPackerImage
ManagedImageId: /subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Compute/images/myPackerImage
ManagedImageLocation: East US

This allows us to do things like add an admin user and generate SSH keys on instance create.

$ az vm create -g packerdemo --name myLinuxDemoMI --image myPackerImage --admin-username azureuser --generate-ssh-keys
SSH key files '/home/builder/.ssh/id_rsa' and '/home/builder/.ssh/id_rsa.pub' have been generated under ~/.ssh to allow SSH access to the VM. If using machines without permanent storage, back up your keys to a safe location.
It is recommended to use parameter "--public-ip-sku Standard" to create new VM with Standard public IP. Please note that the default public IP used for VM creation will be changed from Basic to Standard in the future.
{
  "fqdns": "",
  "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Compute/virtualMachines/myLinuxDemoMI",
  "location": "eastus",
  "macAddress": "00-0D-3A-1E-D4-38",
  "powerState": "VM running",
  "privateIpAddress": "10.0.0.5",
  "publicIpAddress": "20.102.65.87",
  "resourceGroup": "packerdemo",
  "zones": ""
}

And as an aside, if you want to quickly open up web traffic for this instance, that is pretty easy to do as well:

$ az vm open-port --resource-group packerdemo --name myLinuxDemoMI --port 80 --port 443
{
  "defaultSecurityRules": [
    {
      "access": "Allow",
      "description": "Allow inbound traffic from all VMs in VNET",
      "destinationAddressPrefix": "VirtualNetwork",
      "destinationAddressPrefixes": [],
      "destinationApplicationSecurityGroups": null,
      "destinationPortRange": "*",
      "destinationPortRanges": [],
      "direction": "Inbound",
      "etag": "W/\"8e3fac7b-15c8-42fa-8533-7eec6a83246f\"",
      "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Network/networkSecurityGroups/myLinuxDemoMINSG/defaultSecurityRules/AllowVnetInBound",
      "name": "AllowVnetInBound",
      "priority": 65000,
      "protocol": "*",
      "provisioningState": "Succeeded",
      "resourceGroup": "packerdemo",
      "sourceAddressPrefix": "VirtualNetwork",
      "sourceAddressPrefixes": [],
      "sourceApplicationSecurityGroups": null,
      "sourcePortRange": "*",
      "sourcePortRanges": [],
      "type": "Microsoft.Network/networkSecurityGroups/defaultSecurityRules"
    },
    {
      "access": "Allow",
      "description": "Allow inbound traffic from azure load balancer",
      "destinationAddressPrefix": "*",
      "destinationAddressPrefixes": [],
      "destinationApplicationSecurityGroups": null,
      "destinationPortRange": "*",
      "destinationPortRanges": [],
      "direction": "Inbound",
      "etag": "W/\"8e3fac7b-15c8-42fa-8533-7eec6a83246f\"",
      "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Network/networkSecurityGroups/myLinuxDemoMINSG/defaultSecurityRules/AllowAzureLoadBalancerInBound",
      "name": "AllowAzureLoadBalancerInBound",
      "priority": 65001,
      "protocol": "*",
      "provisioningState": "Succeeded",
      "resourceGroup": "packerdemo",
      "sourceAddressPrefix": "AzureLoadBalancer",
      "sourceAddressPrefixes": [],
      "sourceApplicationSecurityGroups": null,
      "sourcePortRange": "*",
      "sourcePortRanges": [],
      "type": "Microsoft.Network/networkSecurityGroups/defaultSecurityRules"
    },
    {
      "access": "Deny",
      "description": "Deny all inbound traffic",
      "destinationAddressPrefix": "*",
      "destinationAddressPrefixes": [],
      "destinationApplicationSecurityGroups": null,
      "destinationPortRange": "*",
      "destinationPortRanges": [],
      "direction": "Inbound",
      "etag": "W/\"8e3fac7b-15c8-42fa-8533-7eec6a83246f\"",
      "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Network/networkSecurityGroups/myLinuxDemoMINSG/defaultSecurityRules/DenyAllInBound",
      "name": "DenyAllInBound",
      "priority": 65500,
      "protocol": "*",
      "provisioningState": "Succeeded",
      "resourceGroup": "packerdemo",
      "sourceAddressPrefix": "*",
      "sourceAddressPrefixes": [],
      "sourceApplicationSecurityGroups": null,
      "sourcePortRange": "*",
      "sourcePortRanges": [],
      "type": "Microsoft.Network/networkSecurityGroups/defaultSecurityRules"
    },
    {
      "access": "Allow",
      "description": "Allow outbound traffic from all VMs to all VMs in VNET",
      "destinationAddressPrefix": "VirtualNetwork",
      "destinationAddressPrefixes": [],
      "destinationApplicationSecurityGroups": null,
      "destinationPortRange": "*",
      "destinationPortRanges": [],
      "direction": "Outbound",
      "etag": "W/\"8e3fac7b-15c8-42fa-8533-7eec6a83246f\"",
      "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Network/networkSecurityGroups/myLinuxDemoMINSG/defaultSecurityRules/AllowVnetOutBound",
      "name": "AllowVnetOutBound",
      "priority": 65000,
      "protocol": "*",
      "provisioningState": "Succeeded",
      "resourceGroup": "packerdemo",
      "sourceAddressPrefix": "VirtualNetwork",
      "sourceAddressPrefixes": [],
      "sourceApplicationSecurityGroups": null,
      "sourcePortRange": "*",
      "sourcePortRanges": [],
      "type": "Microsoft.Network/networkSecurityGroups/defaultSecurityRules"
    },
    {
      "access": "Allow",
      "description": "Allow outbound traffic from all VMs to Internet",
      "destinationAddressPrefix": "Internet",
      "destinationAddressPrefixes": [],
      "destinationApplicationSecurityGroups": null,
      "destinationPortRange": "*",
      "destinationPortRanges": [],
      "direction": "Outbound",
      "etag": "W/\"8e3fac7b-15c8-42fa-8533-7eec6a83246f\"",
      "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Network/networkSecurityGroups/myLinuxDemoMINSG/defaultSecurityRules/AllowInternetOutBound",
      "name": "AllowInternetOutBound",
      "priority": 65001,
      "protocol": "*",
      "provisioningState": "Succeeded",
      "resourceGroup": "packerdemo",
      "sourceAddressPrefix": "*",
      "sourceAddressPrefixes": [],
      "sourceApplicationSecurityGroups": null,
      "sourcePortRange": "*",
      "sourcePortRanges": [],
      "type": "Microsoft.Network/networkSecurityGroups/defaultSecurityRules"
    },
    {
      "access": "Deny",
      "description": "Deny all outbound traffic",
      "destinationAddressPrefix": "*",
      "destinationAddressPrefixes": [],
      "destinationApplicationSecurityGroups": null,
      "destinationPortRange": "*",
      "destinationPortRanges": [],
      "direction": "Outbound",
      "etag": "W/\"8e3fac7b-15c8-42fa-8533-7eec6a83246f\"",
      "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Network/networkSecurityGroups/myLinuxDemoMINSG/defaultSecurityRules/DenyAllOutBound",
      "name": "DenyAllOutBound",
      "priority": 65500,
      "protocol": "*",
      "provisioningState": "Succeeded",
      "resourceGroup": "packerdemo",
      "sourceAddressPrefix": "*",
      "sourceAddressPrefixes": [],
      "sourceApplicationSecurityGroups": null,
      "sourcePortRange": "*",
      "sourcePortRanges": [],
      "type": "Microsoft.Network/networkSecurityGroups/defaultSecurityRules"
    }
  ],
  "etag": "W/\"8e3fac7b-15c8-42fa-8533-7eec6a83246f\"",
  "flowLogs": null,
  "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Network/networkSecurityGroups/myLinuxDemoMINSG",
  "location": "eastus",
  "name": "myLinuxDemoMINSG",
  "networkInterfaces": [
    {
      "dnsSettings": null,
      "dscpConfiguration": null,
      "enableAcceleratedNetworking": null,
      "enableIpForwarding": null,
      "etag": null,
      "extendedLocation": null,
      "hostedWorkloads": null,
      "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Network/networkInterfaces/myLinuxDemoMIVMNic",
      "ipConfigurations": null,
      "location": null,
      "macAddress": null,
      "migrationPhase": null,
      "name": null,
      "networkSecurityGroup": null,
      "nicType": null,
      "primary": null,
      "privateEndpoint": null,
      "privateLinkService": null,
      "provisioningState": null,
      "resourceGroup": "packerdemo",
      "resourceGuid": null,
      "tags": null,
      "tapConfigurations": null,
      "type": null,
      "virtualMachine": null,
      "workloadType": null
    }
  ],
  "provisioningState": "Succeeded",
  "resourceGroup": "packerdemo",
  "resourceGuid": "c84bb553-53d3-460c-b5df-8e53dd65e0dc",
  "securityRules": [
    {
      "access": "Allow",
      "description": null,
      "destinationAddressPrefix": "*",
      "destinationAddressPrefixes": [],
      "destinationApplicationSecurityGroups": null,
      "destinationPortRange": "22",
      "destinationPortRanges": [],
      "direction": "Inbound",
      "etag": "W/\"8e3fac7b-15c8-42fa-8533-7eec6a83246f\"",
      "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Network/networkSecurityGroups/myLinuxDemoMINSG/securityRules/default-allow-ssh",
      "name": "default-allow-ssh",
      "priority": 1000,
      "protocol": "Tcp",
      "provisioningState": "Succeeded",
      "resourceGroup": "packerdemo",
      "sourceAddressPrefix": "*",
      "sourceAddressPrefixes": [],
      "sourceApplicationSecurityGroups": null,
      "sourcePortRange": "*",
      "sourcePortRanges": [],
      "type": "Microsoft.Network/networkSecurityGroups/securityRules"
    },
    {
      "access": "Allow",
      "description": null,
      "destinationAddressPrefix": "*",
      "destinationAddressPrefixes": [],
      "destinationApplicationSecurityGroups": null,
      "destinationPortRange": "443",
      "destinationPortRanges": [],
      "direction": "Inbound",
      "etag": "W/\"8e3fac7b-15c8-42fa-8533-7eec6a83246f\"",
      "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/packerdemo/providers/Microsoft.Network/networkSecurityGroups/myLinuxDemoMINSG/securityRules/open-port-443",
      "name": "open-port-443",
      "priority": 900,
      "protocol": "*",
      "provisioningState": "Succeeded",
      "resourceGroup": "packerdemo",
      "sourceAddressPrefix": "*",
      "sourceAddressPrefixes": [],
      "sourceApplicationSecurityGroups": null,
      "sourcePortRange": "*",
      "sourcePortRanges": [],
      "type": "Microsoft.Network/networkSecurityGroups/securityRules"
    }
  ],
  "subnets": null,
  "tags": {},
  "type": "Microsoft.Network/networkSecurityGroups"
}

Testing

we should just be able to ssh to our host using the admin user we specified and then verify the Azure CLI was installed:

$ ssh azureuser@20.102.65.87
Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 5.4.0-1055-azure x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Mon Sep  6 04:28:41 UTC 2021

  System load:  0.15              Processes:           115
  Usage of /:   8.8% of 28.90GB   Users logged in:     0
  Memory usage: 5%                IP address for eth0: 10.0.0.5
  Swap usage:   0%


37 updates can be applied immediately.
34 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable

New release '20.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.



The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

azureuser@myLinuxDemoMI:~$ az version
{
  "azure-cli": "2.27.2",
  "azure-cli-core": "2.27.2",
  "azure-cli-telemetry": "1.0.6",
  "extensions": {}
}

One thing you will note is that images stored as Managed Images show up as an “Image” type (lower red arrow) compared to page blogs in a storage container (top red arrow): /content/images/2021/09/packer-build-05.png

It was not obvious what the cost would be for this VM image. However, looking at details of managed disk pricing it would appear to be 5c per GB which would be about $1.50, same as the VHD. Assuming the image was 30gb.

We can actually get those details with az image list (az vm image list is for provided images, not ones we created)

$ az image list
[
  {
    "extendedLocation": null,
    "hyperVGeneration": "V1",
    "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/PACKERDEMO/providers/Microsoft.Compute/images/myPackerImage",
    "location": "eastus",
    "name": "myPackerImage",
    "provisioningState": "Succeeded",
    "resourceGroup": "PACKERDEMO",
    "sourceVirtualMachine": {
      "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-u49drt5w46/providers/Microsoft.Compute/virtualMachines/pkrvmu49drt5w46",
      "resourceGroup": "pkr-Resource-Group-u49drt5w46"
    },
    "storageProfile": {
      "dataDisks": [],
      "osDisk": {
        "blobUri": null,
        "caching": "ReadWrite",
        "diskEncryptionSet": null,
        "diskSizeGb": 30,
        "managedDisk": {
          "id": "/subscriptions/944c019d-3e46-422b-b63a-86513f147562/resourceGroups/pkr-Resource-Group-u49drt5w46/providers/Microsoft.Compute/disks/pkrosu49drt5w46",
          "resourceGroup": "pkr-Resource-Group-u49drt5w46"
        },
        "osState": "Generalized",
        "osType": "Linux",
        "snapshot": null,
        "storageAccountType": "Standard_LRS"
      },
      "zoneResilient": false
    },
    "tags": {
      "dept": "engineering"
    },
    "type": "Microsoft.Compute/images"
  }
]

Cleanup

Here we should just need to delete the VM (not also a managed disk).

$ az vm delete -g packerdemo -n myLinuxDemoMI
Are you sure you want to perform this operation? (y/n): y

Even after vm delete, there is a lot of things left around.

One option is to move the completed VM Images to another resource group

/content/images/2021/09/packer-build-06.png

then delete our resource group containing all the left overs (NSGs, Public IPs, etc)

/content/images/2021/09/packer-build-07.png

I’ll delete all of them since these were all just demos.

Summary

In this guide we walked through installing Packer and creating a functional template that builds a Linux VM image in Azure. This image contains the Azure CLI that was setup with a script. We explored the two ways we can create VM images in Azure: VHD and Managed Image.

While the former is being phased out, it is still quite useful for cross-cloud support. For instance we can use AWS VM Import to import into AWS a VM from a VHD we pull down from Azure. We can also easily do the same into GCP or locally with VirtualBox.

In our next blog we will use these VM images for Virtual Machine Scale Sets (VMSS) and Azure Agent Pools amongst other things.

More info:

Wholly unrelated

  • ABBAs first new studio album in 40 years: ABBA Voyage. Suprisingly good.
packer azure hashi getting-started

Isaac Johnson

Isaac Johnson

Cloud Solutions Architect

Isaac is a CSA and DevOps engineer who focuses on cloud migrations and devops processes. He also is a dad to three wonderful daughters (hence the references to Princess King sprinkled throughout the blog).

Theme built by C.S. Rhymes