Fresh/Brewed

GCP Infrastructure Manager with OS Terraform

Published: Sep 26, 2023 by Isaac Johnson

There has been much discussion in the last month about Hashicorp moving from Mozilla Public License v2.0 to the Business Source License (BSL) v1.1 for all future products and some libraries.

None was the vitriol more expressed than with the Terraform community. Terraform, and by extent, Terraform Cloud and Terraform Enterprise only have the size they do because of the OS community and Providers that built and maintained the vast catalogue of modules.

The question has become, what will happen as a result? Will people grumble but stay with it, as reddit still exists? Or will the OS community wholly fork and move on, as they did with Hudson to Jenkins and MySQL to MariaDB?

Perhaps the first major vendor to pivot away from TFE is Google who put out Infrastructure Manager which was announced on Sept 12, 2023. Perhaps this is a basic Terraform Cloud by any other name? Or is it something else? The question becomes what can GCP Infra Manager do? And how does it compare with TF Cloud?

Setup

To use GCP IM, we first need to enable the API.

I use the CLI for this:

$ gcloud auth login
$ gcloud config set project myanthosproject2
Updated property [core/project].

$ gcloud services enable config.googleapis.com
Operation "operations/acat.p2-511842454269-035394c7-9ffb-4df4-902a-94fba0e64dbb" finished successfully.

I’ll want to ensure I have the config.admin role enabled on my user

$ gcloud projects add-iam-policy-binding myanthosproject2 --member="user:isaac.johnson@gmail.com" --role=roles/config.admin
Updated IAM policy for project [myanthosproject2].
bindings:
- members:

I will need an SA for the Infra Manager. Here I’ll create teh InfraManagerService and assing the serviceAccountUser role and config.agent roles:

$ gcloud iam service-accounts create inframgrsvc \
    --description="Infra Manager Service Acction" \
    --display-name="inframgrsvc"

$ gcloud projects add-iam-policy-binding myanthosproject2 \
    --member="serviceAccount:inframgrsvc@myanthosproject2.iam.gserviceaccount.com" \
    --role="roles/iam.serviceAccountUser"
    
$ gcloud projects add-iam-policy-binding myanthosproject2 \
    --member="serviceAccount:inframgrsvc@myanthosproject2.iam.gserviceaccount.com" \
    --role="roles/config.agent"

There is a chance I’ll have to create many more narrow roles (e.g. compute.instance.create) or large big blast radius roles (roles/Owner) so that this SA can actually make things. (see docs)

At this point, my SA can see the bucket we will store the state file in and do basic SA things.

When we look at the Deploy Resources docs we can see many ways to deploy.

I think to start we’ll try a public GIT repo;

gcloud alpha infra-manager deployments apply projects/PROJECT_ID/locations/LOCATION/deployments/DEPLOYMENT_ID \
    --service-account projects/SERVICE_ACCOUNT_PROJECT_ID/serviceAccounts/SERVICE_ACCOUNT \
    --git-source-repo="GIT_REPO" \
    --git-source-directory="DIRECTORY" \
    --git-source-ref="REF" \
    --input-values=INPUT_1_NAME=VALUE,INPUT_2_NAME=VALUE

Let’s keep it simple. Let’s try and and make a bucket

resource "google_storage_bucket" "auto-expire" {
  name          = "auto-expiring-bucket"
  location      = "US"
  force_destroy = true

  lifecycle_rule {
    condition {
      age = 3
    }
    action {
      type = "Delete"
    }
  }

  lifecycle_rule {
    condition {
      age = 1
    }
    action {
      type = "AbortIncompleteMultipartUpload"
    }
  }
}

I put that into a Public GH Repo : https://github.com/idjohnson/gcpimexample/tree/main at SHA 7308216cdac9c60e7d7930ac7fb8e8c231307b99

Will this work? I need a valid location and ID.

$ gcloud alpha infra-manager deployments apply projects/myanthosproject2/locations/us-central1/deployments/main001 \
    --service-account projects/myanthosproject2/serviceAccounts/inframgrsvc@myanthosproject2.iam.gserviceaccount.com \
    --git-source-repo="https://github.com/idjohnson/gcpimexample" \
    --git-source-directory="/" \
    --git-source-ref="7308216cdac9c60e7d7930ac7fb8e8c231307b99"

seems to be starting

builder@LuiGi17:~/Workspaces/jekyll-blog$ gcloud alpha infra-manager deployments apply projects/myanthosproject2/locations/us-central1/deployments/main001 \
    --service-account projects/myanthosproject2/serviceAccounts/inframgrsvc@myanthosproject2.iam.gserviceaccount.com \
    --git-source-repo="https://github.com/idjohnson/gcpimexample" \
    --git-source-directory="/" \
    --git-source-ref="7308216cdac9c60e7d7930ac7fb8e8c231307b99"
Creating the deployment... logs=, step= ...⠛

Though I saw an error:

builder@LuiGi17:~/Workspaces/jekyll-blog$ gcloud alpha infra-manager deployments apply projects/myanthosproject2/locations/us-central1/deployments/main001 \
    --service-account projects/myanthosproject2/serviceAccounts/inframgrsvc@myanthosproject2.iam.gserviceaccount.com \
    --git-source-repo="https://github.com/idjohnson/gcpimexample" \
    --git-source-directory="/" \
    --git-source-ref="7308216cdac9c60e7d7930ac7fb8e8c231307b99"
Creating the deployment... logs=gs://511842454269-us-central1-blueprint-config/main001/r-0/logs ...done.
ERROR: (gcloud.alpha.infra-manager.deployments.apply) Revision failed: The apply build bfac7e7a-b041-4896-a78a-554a3c6c07ae failed while running: tf-apply.
running tf apply: terraform apply failed: running terraform failed: exit status 1.

There is no UI, so we need to pull the log to see what happened

$ gsutil ls gs://511842454269-us-central1-blueprint-config/main001/r-0/logs
gs://511842454269-us-central1-blueprint-config/main001/r-0/logs/log-bfac7e7a-b041-4896-a78a-554a3c6c07ae.txt

$ gsutil cp gs://511842454269-us-central1-blueprint-config/main001/r-0/logs/log-bfac7e7a-b041-4896-a78a-554a3c6c07ae.txt /tmp//log-bfac7e7a-b041-4896-a78a-554a3c6c07ae.txt
Copying gs://511842454269-us-central1-blueprint-config/main001/r-0/logs/log-bfac7e7a-b041-4896-a78a-554a3c6c07ae.txt...
/ [1 files][  7.0 KiB/  7.0 KiB]
Operation completed over 1 objects/7.0 KiB.

$ cat /tmp/log-bfac7e7a-b041-4896-a78a-554a3c6c07ae.txt
starting build "bfac7e7a-b041-4896-a78a-554a3c6c07ae"

FETCHSOURCE
BUILD
Starting Step #0 - "Apply blueprint"
Step #0 - "Apply blueprint": Pulling image: gcr.io/cloud-config-sdk/config-sdk-tf:v0.0.96
Step #0 - "Apply blueprint": v0.0.96: Pulling from cloud-config-sdk/config-sdk-tf
Step #0 - "Apply blueprint": 7264a8db6415: Already exists
Step #0 - "Apply blueprint": 7cc5eab0a257: Pulling fs layer
Step #0 - "Apply blueprint": 8baeb59482eb: Pulling fs layer
Step #0 - "Apply blueprint": 3561c3ad8ba8: Pulling fs layer
Step #0 - "Apply blueprint": d6dbeb1de7c1: Pulling fs layer
Step #0 - "Apply blueprint": 73836d040d72: Pulling fs layer
Step #0 - "Apply blueprint": f232653868ef: Pulling fs layer
Step #0 - "Apply blueprint": 8bf7624adb4e: Pulling fs layer
Step #0 - "Apply blueprint": 290cf887a58d: Pulling fs layer
Step #0 - "Apply blueprint": e789243ebab7: Pulling fs layer
Step #0 - "Apply blueprint": 9c2dc1e39c3b: Pulling fs layer
Step #0 - "Apply blueprint": d6dbeb1de7c1: Waiting
Step #0 - "Apply blueprint": 73836d040d72: Waiting
Step #0 - "Apply blueprint": f232653868ef: Waiting
Step #0 - "Apply blueprint": 8bf7624adb4e: Waiting
Step #0 - "Apply blueprint": 290cf887a58d: Waiting
Step #0 - "Apply blueprint": e789243ebab7: Waiting
Step #0 - "Apply blueprint": 9c2dc1e39c3b: Waiting
Step #0 - "Apply blueprint": 7cc5eab0a257: Download complete
Step #0 - "Apply blueprint": 3561c3ad8ba8: Verifying Checksum
Step #0 - "Apply blueprint": 3561c3ad8ba8: Download complete
Step #0 - "Apply blueprint": 8baeb59482eb: Verifying Checksum
Step #0 - "Apply blueprint": 8baeb59482eb: Download complete
Step #0 - "Apply blueprint": f232653868ef: Download complete
Step #0 - "Apply blueprint": d6dbeb1de7c1: Verifying Checksum
Step #0 - "Apply blueprint": d6dbeb1de7c1: Download complete
Step #0 - "Apply blueprint": 290cf887a58d: Verifying Checksum
Step #0 - "Apply blueprint": 290cf887a58d: Download complete
Step #0 - "Apply blueprint": 8bf7624adb4e: Verifying Checksum
Step #0 - "Apply blueprint": 8bf7624adb4e: Download complete
Step #0 - "Apply blueprint": e789243ebab7: Verifying Checksum
Step #0 - "Apply blueprint": e789243ebab7: Download complete
Step #0 - "Apply blueprint": 7cc5eab0a257: Pull complete
Step #0 - "Apply blueprint": 73836d040d72: Verifying Checksum
Step #0 - "Apply blueprint": 73836d040d72: Download complete
Step #0 - "Apply blueprint": 8baeb59482eb: Pull complete
Step #0 - "Apply blueprint": 9c2dc1e39c3b: Verifying Checksum
Step #0 - "Apply blueprint": 9c2dc1e39c3b: Download complete
Step #0 - "Apply blueprint": 3561c3ad8ba8: Pull complete
Step #0 - "Apply blueprint": d6dbeb1de7c1: Pull complete
Step #0 - "Apply blueprint": 73836d040d72: Pull complete
Step #0 - "Apply blueprint": f232653868ef: Pull complete
Step #0 - "Apply blueprint": 8bf7624adb4e: Pull complete
Step #0 - "Apply blueprint": 290cf887a58d: Pull complete
Step #0 - "Apply blueprint": e789243ebab7: Pull complete
Step #0 - "Apply blueprint": 9c2dc1e39c3b: Pull complete
Step #0 - "Apply blueprint": Digest: sha256:5bb0e68dd41b0d2eedd9a193ea7082fcad3e8cc2ed6825a2d269810bd7db5f04
Step #0 - "Apply blueprint": Status: Downloaded newer image for gcr.io/cloud-config-sdk/config-sdk-tf:v0.0.96
Step #0 - "Apply blueprint": gcr.io/cloud-config-sdk/config-sdk-tf:v0.0.96
Step #0 - "Apply blueprint": time="2023-09-21T22:02:37Z" level=info msg="Logging to GCS object: gs://511842454269-us-central1-blueprint-config/main001/r-0/apply_results/artifacts/log.json"
Step #0 - "Apply blueprint": time="2023-09-21T22:02:37Z" level=info msg="fetching blueprint from git::https://github.com/idjohnson/gcpimexample.git?ref=7308216cdac9c60e7d7930ac7fb8e8c231307b99"
Step #0 - "Apply blueprint": time="2023-09-21T22:02:37Z" level=info msg="Running terraform init in /workspace/apply/content"
Step #0 - "Apply blueprint": Initializing the backend...
Step #0 - "Apply blueprint": Successfully configured the backend "http"! Terraform will automatically
Step #0 - "Apply blueprint": use this backend unless the backend configuration changes.
Step #0 - "Apply blueprint": Initializing provider plugins...
Step #0 - "Apply blueprint": - Finding latest version of hashicorp/google...
Step #0 - "Apply blueprint": - Installing hashicorp/google v4.83.0...
Step #0 - "Apply blueprint": - Installed hashicorp/google v4.83.0 (signed by HashiCorp)
Step #0 - "Apply blueprint": Terraform has created a lock file .terraform.lock.hcl to record the provider
Step #0 - "Apply blueprint": selections it made above. Include this file in your version control repository
Step #0 - "Apply blueprint": so that Terraform can guarantee to make the same selections by default when
Step #0 - "Apply blueprint": you run "terraform init" in the future.
Step #0 - "Apply blueprint": Terraform has been successfully initialized!
Step #0 - "Apply blueprint": time="2023-09-21T22:02:39Z" level=error msg="attribution labels failed for file(s): [no resource supporting labels was found]"
Step #0 - "Apply blueprint": time="2023-09-21T22:02:39Z" level=info msg="Running terraform validate in /workspace/apply/content"
Step #0 - "Apply blueprint": time="2023-09-21T22:02:39Z" level=info msg="Running terraform plan in /workspace/apply/content"
Step #0 - "Apply blueprint": Terraform 1.2.3
Step #0 - "Apply blueprint": Terraform 1.2.3
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Plan to create
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Plan to create
Step #0 - "Apply blueprint": Plan: 1 to add, 0 to change, 0 to destroy.
Step #0 - "Apply blueprint": time="2023-09-21T22:02:41Z" level=info msg="Running terraform show in /workspace/apply/content on /tmp/tfplan-2804650961/plan.out"
Step #0 - "Apply blueprint": Plan: 1 to add, 0 to change, 0 to destroy.
Step #0 - "Apply blueprint": time="2023-09-21T22:02:42Z" level=info msg="Running terraform apply in /workspace/apply/content with /tmp/tfplan-2804650961/plan.out plan"
Step #0 - "Apply blueprint": Terraform 1.2.3
Step #0 - "Apply blueprint": Terraform 1.2.3
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Plan to create
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Plan to create
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Creating...
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Creating...
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Creation errored after 0s
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Creation errored after 0s
Step #0 - "Apply blueprint": time="2023-09-21T22:02:44Z" level=error msg="Error (exit code 1) running \"terraform apply -json /tmp/tfplan-2804650961/plan.out\". Stderr:\n"
Step #0 - "Apply blueprint": Error: project: required field is not set
Step #0 - "Apply blueprint": Error: project: required field is not set
Step #0 - "Apply blueprint": running tf apply: terraform apply failed: running terraform failed: exit status 1
Step #0 - "Apply blueprint": Error: running tf apply: terraform apply failed: running terraform failed: exit status 1
Finished Step #0 - "Apply blueprint"
ERROR
ERROR: build step 0 "gcr.io/cloud-config-sdk/config-sdk-tf:v0.0.96" failed: step exited with non-zero status: 1

I added a project (fixed for the moment) to github.

resource "google_storage_bucket" "auto-expire" {
  name          = "auto-expiring-bucket"
  location      = "US"
  project       = "myanthosproject2"
  force_destroy = true

  lifecycle_rule {
    condition {
      age = 3
    }
    action {
      type = "Delete"
    }
  }

  lifecycle_rule {
    condition {
      age = 1
    }
    action {
      type = "AbortIncompleteMultipartUpload"
    }
  }
}

Then ran again

$ gcloud alpha infra-manager deployments apply projects/myanthosproject2/locations/us-central1/deployments/main004 --service-account projects/myanthosproject2/serviceAccounts/inframgrsvc@myanthosproject2.iam.gserviceaccount.com --git-source-repo="https://github.com/idjohnson/gcpimexample" --git-source-directory="/" --git-source-ref="ad85d9748f6d9f1ee838dc2d404c7df02283593c"
Creating the deployment... logs=gs://511842454269-us-central1-blueprint-config/main004/r-0/logs, step=RUNNING_TF_PLAN ...done.
ERROR: (gcloud.alpha.infra-manager.deployments.apply) Revision failed: The apply build 9236cf72-8920-40a8-8ed5-99ac0fbb6e5b failed while running: tf-apply.
running tf apply: terraform apply failed: running terraform failed: exit status 1.

$ gsutil ls gs://511842454269-us-central1-blueprint-config/main004/r-0/logs
gs://511842454269-us-central1-blueprint-config/main004/r-0/logs/log-9236cf72-8920-40a8-8ed5-99ac0fbb6e5b.txt

$ gsutil cp gs://511842454269-us-central1-blueprint-config/main004/r-0/logs/log-9236cf72-8920-40a8-8ed5-99ac0fbb6e5b.txt /tmp/log-9236cf72-8920-40a8-8ed5-99ac0fbb6e5b.txt
Copying gs://511842454269-us-central1-blueprint-config/main004/r-0/logs/log-9236cf72-8920-40a8-8ed5-99ac0fbb6e5b.txt...
/ [1 files][  7.4 KiB/  7.4 KiB]
Operation completed over 1 objects/7.4 KiB.

$ cat /tmp/log-9236cf72-8920-40a8-8ed5-99ac0fbb6e5b.txt
starting build "9236cf72-8920-40a8-8ed5-99ac0fbb6e5b"

FETCHSOURCE
BUILD
Starting Step #0 - "Apply blueprint"
Step #0 - "Apply blueprint": Pulling image: gcr.io/cloud-config-sdk/config-sdk-tf:v0.0.96
Step #0 - "Apply blueprint": v0.0.96: Pulling from cloud-config-sdk/config-sdk-tf
Step #0 - "Apply blueprint": 7264a8db6415: Already exists
Step #0 - "Apply blueprint": 7cc5eab0a257: Pulling fs layer
Step #0 - "Apply blueprint": 8baeb59482eb: Pulling fs layer
Step #0 - "Apply blueprint": 3561c3ad8ba8: Pulling fs layer
Step #0 - "Apply blueprint": d6dbeb1de7c1: Pulling fs layer
Step #0 - "Apply blueprint": 73836d040d72: Pulling fs layer
Step #0 - "Apply blueprint": f232653868ef: Pulling fs layer
Step #0 - "Apply blueprint": 8bf7624adb4e: Pulling fs layer
Step #0 - "Apply blueprint": 290cf887a58d: Pulling fs layer
Step #0 - "Apply blueprint": e789243ebab7: Pulling fs layer
Step #0 - "Apply blueprint": 9c2dc1e39c3b: Pulling fs layer
Step #0 - "Apply blueprint": d6dbeb1de7c1: Waiting
Step #0 - "Apply blueprint": 73836d040d72: Waiting
Step #0 - "Apply blueprint": f232653868ef: Waiting
Step #0 - "Apply blueprint": 8bf7624adb4e: Waiting
Step #0 - "Apply blueprint": 290cf887a58d: Waiting
Step #0 - "Apply blueprint": e789243ebab7: Waiting
Step #0 - "Apply blueprint": 9c2dc1e39c3b: Waiting
Step #0 - "Apply blueprint": 7cc5eab0a257: Verifying Checksum
Step #0 - "Apply blueprint": 7cc5eab0a257: Download complete
Step #0 - "Apply blueprint": 8baeb59482eb: Verifying Checksum
Step #0 - "Apply blueprint": 8baeb59482eb: Download complete
Step #0 - "Apply blueprint": 3561c3ad8ba8: Verifying Checksum
Step #0 - "Apply blueprint": 3561c3ad8ba8: Download complete
Step #0 - "Apply blueprint": d6dbeb1de7c1: Verifying Checksum
Step #0 - "Apply blueprint": d6dbeb1de7c1: Download complete
Step #0 - "Apply blueprint": f232653868ef: Verifying Checksum
Step #0 - "Apply blueprint": f232653868ef: Download complete
Step #0 - "Apply blueprint": 8bf7624adb4e: Download complete
Step #0 - "Apply blueprint": 290cf887a58d: Verifying Checksum
Step #0 - "Apply blueprint": 290cf887a58d: Download complete
Step #0 - "Apply blueprint": e789243ebab7: Verifying Checksum
Step #0 - "Apply blueprint": e789243ebab7: Download complete
Step #0 - "Apply blueprint": 7cc5eab0a257: Pull complete
Step #0 - "Apply blueprint": 73836d040d72: Verifying Checksum
Step #0 - "Apply blueprint": 73836d040d72: Download complete
Step #0 - "Apply blueprint": 8baeb59482eb: Pull complete
Step #0 - "Apply blueprint": 9c2dc1e39c3b: Verifying Checksum
Step #0 - "Apply blueprint": 9c2dc1e39c3b: Download complete
Step #0 - "Apply blueprint": 3561c3ad8ba8: Pull complete
Step #0 - "Apply blueprint": d6dbeb1de7c1: Pull complete
Step #0 - "Apply blueprint": 73836d040d72: Pull complete
Step #0 - "Apply blueprint": f232653868ef: Pull complete
Step #0 - "Apply blueprint": 8bf7624adb4e: Pull complete
Step #0 - "Apply blueprint": 290cf887a58d: Pull complete
Step #0 - "Apply blueprint": e789243ebab7: Pull complete
Step #0 - "Apply blueprint": 9c2dc1e39c3b: Pull complete
Step #0 - "Apply blueprint": Digest: sha256:5bb0e68dd41b0d2eedd9a193ea7082fcad3e8cc2ed6825a2d269810bd7db5f04
Step #0 - "Apply blueprint": Status: Downloaded newer image for gcr.io/cloud-config-sdk/config-sdk-tf:v0.0.96
Step #0 - "Apply blueprint": gcr.io/cloud-config-sdk/config-sdk-tf:v0.0.96
Step #0 - "Apply blueprint": time="2023-09-21T23:38:03Z" level=info msg="Logging to GCS object: gs://511842454269-us-central1-blueprint-config/main004/r-0/apply_results/artifacts/log.json"
Step #0 - "Apply blueprint": time="2023-09-21T23:38:03Z" level=info msg="fetching blueprint from git::https://github.com/idjohnson/gcpimexample.git?ref=ad85d9748f6d9f1ee838dc2d404c7df02283593c"
Step #0 - "Apply blueprint": time="2023-09-21T23:38:03Z" level=info msg="Running terraform init in /workspace/apply/content"
Step #0 - "Apply blueprint": Initializing the backend...
Step #0 - "Apply blueprint": Successfully configured the backend "http"! Terraform will automatically
Step #0 - "Apply blueprint": use this backend unless the backend configuration changes.
Step #0 - "Apply blueprint": Initializing provider plugins...
Step #0 - "Apply blueprint": - Finding latest version of hashicorp/google...
Step #0 - "Apply blueprint": - Installing hashicorp/google v4.83.0...
Step #0 - "Apply blueprint": - Installed hashicorp/google v4.83.0 (signed by HashiCorp)
Step #0 - "Apply blueprint": Terraform has created a lock file .terraform.lock.hcl to record the provider
Step #0 - "Apply blueprint": selections it made above. Include this file in your version control repository
Step #0 - "Apply blueprint": so that Terraform can guarantee to make the same selections by default when
Step #0 - "Apply blueprint": you run "terraform init" in the future.
Step #0 - "Apply blueprint": Terraform has been successfully initialized!
Step #0 - "Apply blueprint": time="2023-09-21T23:38:05Z" level=error msg="attribution labels failed for file(s): [no resource supporting labels was found]"
Step #0 - "Apply blueprint": time="2023-09-21T23:38:05Z" level=info msg="Running terraform validate in /workspace/apply/content"
Step #0 - "Apply blueprint": time="2023-09-21T23:38:06Z" level=info msg="Running terraform plan in /workspace/apply/content"
Step #0 - "Apply blueprint": Terraform 1.2.3
Step #0 - "Apply blueprint": Terraform 1.2.3
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Plan to create
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Plan to create
Step #0 - "Apply blueprint": Plan: 1 to add, 0 to change, 0 to destroy.
Step #0 - "Apply blueprint": Plan: 1 to add, 0 to change, 0 to destroy.
Step #0 - "Apply blueprint": time="2023-09-21T23:38:08Z" level=info msg="Running terraform show in /workspace/apply/content on /tmp/tfplan-1841324626/plan.out"
Step #0 - "Apply blueprint": time="2023-09-21T23:38:09Z" level=info msg="Running terraform apply in /workspace/apply/content with /tmp/tfplan-1841324626/plan.out plan"
Step #0 - "Apply blueprint": Terraform 1.2.3
Step #0 - "Apply blueprint": Terraform 1.2.3
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Plan to create
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Plan to create
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Creating...
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Creating...
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Creation errored after 1s
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Creation errored after 1s
Step #0 - "Apply blueprint": time="2023-09-21T23:38:11Z" level=error msg="Error (exit code 1) running \"terraform apply -json /tmp/tfplan-1841324626/plan.out\". Stderr:\n"
Step #0 - "Apply blueprint": Error: googleapi: Error 409: The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again., conflict
Step #0 - "Apply blueprint": Error: googleapi: Error 409: The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again., conflict
Step #0 - "Apply blueprint": running tf apply: terraform apply failed: running terraform failed: exit status 1
Step #0 - "Apply blueprint": Error: running tf apply: terraform apply failed: running terraform failed: exit status 1
Finished Step #0 - "Apply blueprint"
ERROR
ERROR: build step 0 "gcr.io/cloud-config-sdk/config-sdk-tf:v0.0.96" failed: step exited with non-zero status: 1

Ah, that is a simple fix. Let’s use a unique name. Actually, I’ll put in some vars I can use from the command line

resource "google_storage_bucket" "auto-expire" {
  name          = var.bucketname
  location      = "US"
  project       = var.projectid
  force_destroy = true

  lifecycle_rule {
    condition {
      age = 3
    }
    action {
      type = "Delete"
    }
  }

  lifecycle_rule {
    condition {
      age = 1
    }
    action {
      type = "AbortIncompleteMultipartUpload"
    }
  }
}

Applying generated another error

$ gcloud alpha infra-manager deployments apply projects/myanthosproject2/locations/us-central1/deployments/main006 --service-account projects/myanthosproject2/serviceAccounts/inframgrsvc@myanthosproject2.iam.gserviceaccount.com --git-source-repo="https://github.com/idjohnson/gcpimexample" --git-source-directory="/" --git-source-ref="11063aa" --input-values=projectid=myanthosproject2,bucketname=fbstestbucket123dd321
Creating the deployment... logs=gs://511842454269-us-central1-blueprint-config/main006/r-0/logs ...done.
ERROR: (gcloud.alpha.infra-manager.deployments.apply) Revision failed: The apply build bc9a3a99-a6de-424c-a30b-6de88cd5bf62 failed while running: tf-validate.
running tf validate: terraform validate failed: running terraform failed: exit status 1.

Pulling the log reminded me I need to declare variables. Doh!

...
Step #0 - "Apply blueprint": Terraform has been successfully initialized!
Step #0 - "Apply blueprint": time="2023-09-21T23:52:28Z" level=error msg="attribution labels failed for file(s): [no resource supporting labels was found]"
Step #0 - "Apply blueprint": time="2023-09-21T23:52:28Z" level=info msg="Running terraform validate in /workspace/apply/content"
Step #0 - "Apply blueprint": time="2023-09-21T23:52:29Z" level=error msg="Error (exit code 1) running \"terraform validate -json\". Stderr:\n"
Step #0 - "Apply blueprint": Found 2 errors and 0 warnings.
Step #0 - "Apply blueprint": error: Reference to undeclared input variable
Step #0 - "Apply blueprint": An input variable with the name "projectid" has not been declared. This variable can be declared with a variable "projectid" {} block.
Step #0 - "Apply blueprint": on buckets.tf line 4
Step #0 - "Apply blueprint":
Step #0 - "Apply blueprint":
Step #0 - "Apply blueprint":
Step #0 - "Apply blueprint": error: Reference to undeclared input variable
Step #0 - "Apply blueprint": An input variable with the name "bucketname" has not been declared. This variable can be declared with a variable "bucketname" {} block.
Step #0 - "Apply blueprint": on buckets.tf line 2
Step #0 - "Apply blueprint":
Step #0 - "Apply blueprint":
Step #0 - "Apply blueprint":
Step #0 - "Apply blueprint": running tf validate: terraform validate failed: running terraform failed: exit status 1
Step #0 - "Apply blueprint": Error: running tf validate: terraform validate failed: running terraform failed: exit status 1
Finished Step #0 - "Apply blueprint"
ERROR
ERROR: build step 0 "gcr.io/cloud-config-sdk/config-sdk-tf:v0.0.96" failed: step exited with non-zero status: 1

I’ll create a new variables file

/content/images/2023/09/gcpimframgr-01.png

variable "projectid" {
  type        = string
  description = "The name of the GCP project"
}

variable "bucketname" {
  type        = string
  description = "The name of the bucket to try and create"
}

This time it worked!

$ gcloud alpha infra-manager deployments apply projects/myanthosproject2/locations/us-central1/deployments/main007 --service-account projects/myanthosproject2/serviceAccounts/inframgrsvc@myanthosproject2.iam.gserviceaccount.com --git-source-repo="https://github.com/idjohnson/gcpimexample" --git-source-directory="/" --git-source-ref="5031a4c27658a8a76695120fdd3973f0376569b6" --input-values=projectid=myanthosproject2,bucketname=fbstestbucket123dd321
Creating the deployment... logs=gs://511842454269-us-central1-blueprint-config/main007/r-0/logs ...done.

We can check the log

builder@LuiGi17:~/Workspaces/jekyll-blog$ gsutil ls gs://511842454269-us-central1-blueprint-config/main007/r-0/logs
gs://511842454269-us-central1-blueprint-config/main007/r-0/logs/log-8cc7671e-b29a-45c7-bac8-dbb34c6eafe8.txt
builder@LuiGi17:~/Workspaces/jekyll-blog$ gsutil cp gs://511842454269-us-central1-blueprint-config/main007/r-0/logs/log-8cc7671e-b29a-45c7-bac8-dbb34c6eafe8.txt /tmp/log-8cc7671e-b29a-45c7-bac8-dbb34c6eafe8.txt
Copying gs://511842454269-us-central1-blueprint-config/main007/r-0/logs/log-8cc7671e-b29a-45c7-bac8-dbb34c6eafe8.txt...
/ [1 files][  7.2 KiB/  7.2 KiB]
Operation completed over 1 objects/7.2 KiB.
builder@LuiGi17:~/Workspaces/jekyll-blog$ cat /tmp/log-8cc7671e-b29a-45c7-bac8-dbb34c6eafe8.txt
starting build "8cc7671e-b29a-45c7-bac8-dbb34c6eafe8"

FETCHSOURCE
BUILD
Starting Step #0 - "Apply blueprint"
Step #0 - "Apply blueprint": Pulling image: gcr.io/cloud-config-sdk/config-sdk-tf:v0.0.96
Step #0 - "Apply blueprint": v0.0.96: Pulling from cloud-config-sdk/config-sdk-tf
Step #0 - "Apply blueprint": 7264a8db6415: Already exists
Step #0 - "Apply blueprint": 7cc5eab0a257: Pulling fs layer
Step #0 - "Apply blueprint": 8baeb59482eb: Pulling fs layer
Step #0 - "Apply blueprint": 3561c3ad8ba8: Pulling fs layer
Step #0 - "Apply blueprint": d6dbeb1de7c1: Pulling fs layer
Step #0 - "Apply blueprint": 73836d040d72: Pulling fs layer
Step #0 - "Apply blueprint": f232653868ef: Pulling fs layer
Step #0 - "Apply blueprint": 8bf7624adb4e: Pulling fs layer
Step #0 - "Apply blueprint": 290cf887a58d: Pulling fs layer
Step #0 - "Apply blueprint": e789243ebab7: Pulling fs layer
Step #0 - "Apply blueprint": 9c2dc1e39c3b: Pulling fs layer
Step #0 - "Apply blueprint": d6dbeb1de7c1: Waiting
Step #0 - "Apply blueprint": 73836d040d72: Waiting
Step #0 - "Apply blueprint": f232653868ef: Waiting
Step #0 - "Apply blueprint": 8bf7624adb4e: Waiting
Step #0 - "Apply blueprint": 290cf887a58d: Waiting
Step #0 - "Apply blueprint": e789243ebab7: Waiting
Step #0 - "Apply blueprint": 9c2dc1e39c3b: Waiting
Step #0 - "Apply blueprint": 7cc5eab0a257: Verifying Checksum
Step #0 - "Apply blueprint": 7cc5eab0a257: Download complete
Step #0 - "Apply blueprint": d6dbeb1de7c1: Verifying Checksum
Step #0 - "Apply blueprint": d6dbeb1de7c1: Download complete
Step #0 - "Apply blueprint": 8baeb59482eb: Download complete
Step #0 - "Apply blueprint": 3561c3ad8ba8: Verifying Checksum
Step #0 - "Apply blueprint": 3561c3ad8ba8: Download complete
Step #0 - "Apply blueprint": f232653868ef: Verifying Checksum
Step #0 - "Apply blueprint": f232653868ef: Download complete
Step #0 - "Apply blueprint": 8bf7624adb4e: Verifying Checksum
Step #0 - "Apply blueprint": 8bf7624adb4e: Download complete
Step #0 - "Apply blueprint": 290cf887a58d: Verifying Checksum
Step #0 - "Apply blueprint": 290cf887a58d: Download complete
Step #0 - "Apply blueprint": e789243ebab7: Verifying Checksum
Step #0 - "Apply blueprint": e789243ebab7: Download complete
Step #0 - "Apply blueprint": 7cc5eab0a257: Pull complete
Step #0 - "Apply blueprint": 73836d040d72: Verifying Checksum
Step #0 - "Apply blueprint": 73836d040d72: Download complete
Step #0 - "Apply blueprint": 8baeb59482eb: Pull complete
Step #0 - "Apply blueprint": 9c2dc1e39c3b: Download complete
Step #0 - "Apply blueprint": 3561c3ad8ba8: Pull complete
Step #0 - "Apply blueprint": d6dbeb1de7c1: Pull complete
Step #0 - "Apply blueprint": 73836d040d72: Pull complete
Step #0 - "Apply blueprint": f232653868ef: Pull complete
Step #0 - "Apply blueprint": 8bf7624adb4e: Pull complete
Step #0 - "Apply blueprint": 290cf887a58d: Pull complete
Step #0 - "Apply blueprint": e789243ebab7: Pull complete
Step #0 - "Apply blueprint": 9c2dc1e39c3b: Pull complete
Step #0 - "Apply blueprint": Digest: sha256:5bb0e68dd41b0d2eedd9a193ea7082fcad3e8cc2ed6825a2d269810bd7db5f04
Step #0 - "Apply blueprint": Status: Downloaded newer image for gcr.io/cloud-config-sdk/config-sdk-tf:v0.0.96
Step #0 - "Apply blueprint": gcr.io/cloud-config-sdk/config-sdk-tf:v0.0.96
Step #0 - "Apply blueprint": time="2023-09-21T23:57:44Z" level=info msg="Logging to GCS object: gs://511842454269-us-central1-blueprint-config/main007/r-0/apply_results/artifacts/log.json"
Step #0 - "Apply blueprint": time="2023-09-21T23:57:44Z" level=info msg="fetching blueprint from git::https://github.com/idjohnson/gcpimexample.git?ref=5031a4c27658a8a76695120fdd3973f0376569b6"
Step #0 - "Apply blueprint": time="2023-09-21T23:57:45Z" level=info msg="Running terraform init in /workspace/apply/content"
Step #0 - "Apply blueprint": Initializing the backend...
Step #0 - "Apply blueprint": Successfully configured the backend "http"! Terraform will automatically
Step #0 - "Apply blueprint": use this backend unless the backend configuration changes.
Step #0 - "Apply blueprint": Initializing provider plugins...
Step #0 - "Apply blueprint": - Finding latest version of hashicorp/google...
Step #0 - "Apply blueprint": - Installing hashicorp/google v4.83.0...
Step #0 - "Apply blueprint": - Installed hashicorp/google v4.83.0 (signed by HashiCorp)
Step #0 - "Apply blueprint": Terraform has created a lock file .terraform.lock.hcl to record the provider
Step #0 - "Apply blueprint": selections it made above. Include this file in your version control repository
Step #0 - "Apply blueprint": so that Terraform can guarantee to make the same selections by default when
Step #0 - "Apply blueprint": you run "terraform init" in the future.
Step #0 - "Apply blueprint": Terraform has been successfully initialized!
Step #0 - "Apply blueprint": time="2023-09-21T23:57:47Z" level=error msg="attribution labels failed for file(s): [no resource supporting labels was found no resource supporting labels was found]"
Step #0 - "Apply blueprint": time="2023-09-21T23:57:47Z" level=info msg="Running terraform validate in /workspace/apply/content"
Step #0 - "Apply blueprint": time="2023-09-21T23:57:48Z" level=info msg="Running terraform plan in /workspace/apply/content"
Step #0 - "Apply blueprint": Terraform 1.2.3
Step #0 - "Apply blueprint": Terraform 1.2.3
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Plan to create
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Plan to create
Step #0 - "Apply blueprint": Plan: 1 to add, 0 to change, 0 to destroy.
Step #0 - "Apply blueprint": time="2023-09-21T23:57:50Z" level=info msg="Running terraform show in /workspace/apply/content on /tmp/tfplan-3823449154/plan.out"
Step #0 - "Apply blueprint": Plan: 1 to add, 0 to change, 0 to destroy.
Step #0 - "Apply blueprint": time="2023-09-21T23:57:51Z" level=info msg="Running terraform apply in /workspace/apply/content with /tmp/tfplan-3823449154/plan.out plan"
Step #0 - "Apply blueprint": Terraform 1.2.3
Step #0 - "Apply blueprint": Terraform 1.2.3
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Plan to create
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Plan to create
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Creating...
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Creating...
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Creation complete after 1s [id=fbstestbucket123dd321]
Step #0 - "Apply blueprint": google_storage_bucket.auto-expire: Creation complete after 1s [id=fbstestbucket123dd321]
Step #0 - "Apply blueprint": Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Step #0 - "Apply blueprint": Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Step #0 - "Apply blueprint": Outputs: 0
Step #0 - "Apply blueprint": Outputs: 0
Step #0 - "Apply blueprint": time="2023-09-21T23:57:53Z" level=info msg="Running terraform output in /workspace/apply/content"
Step #0 - "Apply blueprint": time="2023-09-21T23:57:54Z" level=info msg="Running terraform show in /workspace/apply/content"
Finished Step #0 - "Apply blueprint"
Starting Step #1
Step #1: Already have image (with digest): gcr.io/cloud-config-sdk/config-sdk-tf:v0.0.96
Step #1:
Step #1:
Step #1: Build Artifacts: gs://511842454269-us-central1-blueprint-config/main007/r-0/apply_results/content
Finished Step #1
PUSH
DONE

I can see the bucket was created

/content/images/2023/09/gcpimframgr-02.png

and more importantly, I can check that the lifecycle policy we defined was also applied

/content/images/2023/09/gcpimframgr-03.png

Listing Deployments

We can see a list of all our runs using the CLI

$ gcloud alpha infra-manager deployments list --project myanthosproject2 --location us-central1
NAME     STATE   LATEST_REVISION  CREATE_TIME                     UPDATE_TIME
main001  FAILED  r-1              2023-09-21T22:02:20.390607026Z  2023-09-21T23:25:45.199173024Z
main002  FAILED  r-0              2023-09-21T23:26:47.876137433Z  2023-09-21T23:27:24.270210040Z
main003  FAILED  r-0              2023-09-21T23:32:36.767666544Z  2023-09-21T23:33:07.307129425Z
main004  FAILED  r-0              2023-09-21T23:37:49.265017193Z  2023-09-21T23:38:20.822931830Z
main005  FAILED  r-1              2023-09-21T23:49:54.920990616Z  2023-09-21T23:51:24.389083043Z
main006  FAILED  r-0              2023-09-21T23:52:08.002179577Z  2023-09-21T23:52:34.910674620Z
main007  ACTIVE  r-0              2023-09-21T23:57:31.236443752Z  2023-09-21T23:58:07.837383467Z

I realize now I could have saved myself a lot of trouble manually copying logs locally to view, just so I could see errors.

We can see details on past failed deployments

$ gcloud alpha infra-manager deployments describe projects/myanthosproject2/locations/us-central1/deployments/main006
createTime: '2023-09-21T23:52:08.002179577Z'
errorCode: REVISION_FAILED
errorLogs: gs://511842454269-us-central1-blueprint-config/main006/r-0/errors/tf-error.ndjson
latestRevision: projects/myanthosproject2/locations/us-central1/deployments/main006/revisions/r-0
lockState: UNLOCKED
name: projects/myanthosproject2/locations/us-central1/deployments/main006
serviceAccount: projects/myanthosproject2/serviceAccounts/inframgrsvc@myanthosproject2.iam.gserviceaccount.com
state: FAILED
stateDetail: |-
  Revision failed: The apply build bc9a3a99-a6de-424c-a30b-6de88cd5bf62 failed while running: tf-validate.
  running tf validate: terraform validate failed: running terraform failed: exit status 1.
terraformBlueprint:
  gitSource:
    directory: /
    ref: 11063aa
    repo: https://github.com/idjohnson/gcpimexample
  inputValues:
    bucketname:
      inputValue: fbstestbucket123dd321
    projectid:
      inputValue: myanthosproject2
tfErrors:
- errorDescription: Reference to undeclared input variable. An input variable with
    the name "projectid" has not been declared. This variable can be declared with
    a variable "projectid" {} block.
  resourceAddress: resource "google_storage_bucket" "auto-expire"
- errorDescription: Reference to undeclared input variable. An input variable with
    the name "bucketname" has not been declared. This variable can be declared with
    a variable "bucketname" {} block.
  resourceAddress: resource "google_storage_bucket" "auto-expire"
updateTime: '2023-09-21T23:52:34.910674620Z'

as well as a successful one

$ gcloud alpha infra-manager deployments describe projects/myanthosproject2/locations/us-central1/deployments/main007
createTime: '2023-09-21T23:57:31.236443752Z'
latestRevision: projects/myanthosproject2/locations/us-central1/deployments/main007/revisions/r-0
lockState: UNLOCKED
name: projects/myanthosproject2/locations/us-central1/deployments/main007
serviceAccount: projects/myanthosproject2/serviceAccounts/inframgrsvc@myanthosproject2.iam.gserviceaccount.com
state: ACTIVE
stateDetail: revision "projects/myanthosproject2/locations/us-central1/deployments/main007/revisions/r-0"
  applied
terraformBlueprint:
  gitSource:
    directory: /
    ref: 5031a4c27658a8a76695120fdd3973f0376569b6
    repo: https://github.com/idjohnson/gcpimexample
  inputValues:
    bucketname:
      inputValue: fbstestbucket123dd321
    projectid:
      inputValue: myanthosproject2
updateTime: '2023-09-21T23:58:07.837383467Z'

We can also look at iterations on a run. For instance, I ran it twice with different params on run main005

builder@LuiGi17:~/Workspaces/jekyll-blog$ gcloud alpha infra-manager revisions list --deployment=projects/myanthosproject2/locations/us-central1/deployments/main007
NAME  STATE    CREATE_TIME                     UPDATE_TIME
r-0   APPLIED  2023-09-21T23:57:31.929503120Z  2023-09-21T23:58:03.334732559Z
builder@LuiGi17:~/Workspaces/jekyll-blog$ gcloud alpha infra-manager revisions list --deployment=projects/myanthosproject2/locations/us-central1/deployments/main005
NAME  STATE   CREATE_TIME                     UPDATE_TIME
r-0   FAILED  2023-09-21T23:49:55.716564813Z  2023-09-21T23:50:21.503013471Z
r-1   FAILED  2023-09-21T23:51:01.433937511Z  2023-09-21T23:51:22.340708770Z

And I can use that first revision in the Revisions Describe command to see details of that run

$ gcloud alpha infra-manager revisions describe projects/myanthosproject2/locations/us-central1/deployments/main005/revisions/r-0
action: CREATE
applyResults:
  artifacts: gs://511842454269-us-central1-blueprint-config/main005/r-0/apply_results/artifacts
  content: gs://511842454269-us-central1-blueprint-config/main005/r-0/apply_results/content
build: d437b907-b2eb-459e-a0d5-bdcc3f122903
createTime: '2023-09-21T23:49:55.716564813Z'
errorCode: APPLY_BUILD_RUN_FAILED
errorLogs: gs://511842454269-us-central1-blueprint-config/main005/r-0/errors/tf-error.ndjson
logs: gs://511842454269-us-central1-blueprint-config/main005/r-0/logs
name: projects/myanthosproject2/locations/us-central1/deployments/main005/revisions/r-0
serviceAccount: projects/myanthosproject2/serviceAccounts/inframgrsvc@myanthosproject2.iam.gserviceaccount.com
state: FAILED
stateDetail: |-
  The apply build d437b907-b2eb-459e-a0d5-bdcc3f122903 failed while running: tf-validate.
  running tf validate: terraform validate failed: running terraform failed: exit status 1.
terraformBlueprint:
  gitSource:
    directory: /
    ref: 11063aa
    repo: https://github.com/idjohnson/gcpimexample
  inputValues:
    bucketid:
      inputValue: fbstestbucket123321
    projectid:
      inputValue: myanthosproject2
tfErrors:
- errorDescription: Reference to undeclared input variable. An input variable with
    the name "projectid" has not been declared. This variable can be declared with
    a variable "projectid" {} block.
  resourceAddress: resource "google_storage_bucket" "auto-expire"
- errorDescription: Reference to undeclared input variable. An input variable with
    the name "bucketname" has not been declared. This variable can be declared with
    a variable "bucketname" {} block.
  resourceAddress: resource "google_storage_bucket" "auto-expire"
updateTime: '2023-09-21T23:50:21.503013471Z'

I can view the resouces created in a revision

$ gcloud alpha infra-manager resources list --revision=projects/myanthosproject2/locations/us-central1/deployments/main007/revisions/r-0
NAME                     STATE
storage-bucket-5u4auyu4  RECONCILED

which we can feed into the describe command to view the actual details, in this case, the bucket name

$ gcloud alpha infra-manager resources describe projects/myanthosproject2/locations/us-central1/deployments/main007/revisions/r-0/resources/storage-bucket-5u4auyu4
caiAssets:
  storage.googleapis.com/Bucket:
    fullResourceName: //storage.googleapis.com/fbstestbucket123dd321
intent: CREATE
name: projects/myanthosproject2/locations/us-central1/deployments/main007/revisions/r-0/resources/storage-bucket-5u4auyu4
state: RECONCILED
terraformInfo:
  address: google_storage_bucket.auto-expire
  id: fbstestbucket123dd321

Getting state file.

There are a lot of details you can see in manually managing the state file here in docs

But essentially, we can pull down the state file of a deployment

$ gcloud alpha infra-manager deployments export-statefile main007 --project myanthosproject2 --location us-central1 --format="ge
t(signedUri)"
Initiating export state file request...
https://storage.googleapis.com/511842454269-us-central1-tf-states/main007/default.tfstate?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=cloud-control2-config-clh%40system.gserviceaccount.com%2F20230922%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20230922T001229Z&X-Goog-Expires=299&X-Goog-Signature=4e4dfef33bcdd1cd6090fba6635c060f83d1deab0ba369ffb73af62d59f8744f688eb7f21dxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxcc4eaf9475697d4ab68f06136707f22db521fdb55a61e52d6ccbc00a7&X-Goog-SignedHeaders=host

# Now Download
$ curl -s -X GET --output terraform.tfstate 'https://storage.googleapis.com/511842454269-us-central1-tf-states/main007/default.tfstate?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=cloud-control2-config-clh%40system.gserviceaccount.com%2F20230922%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20230922T001229Z&X-Goog-Expires=299&X-Goog-Signature=4e4dfef33bcdd1cd6090fba6635c060f83d1deab0ba369ffb73af62d59f8744f688eb7f21dxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx5697d4ab68f06136707f22db521fdb55a61e52d6ccbc00a7&X-Goog-SignedHeaders=host'

And I can view the state

$ cat terraform.tfstate
{
  "version": 4,
  "terraform_version": "1.2.3",
  "serial": 0,
  "lineage": "b4a58e8b-742f-7453-b758-1cf4da5bf4e2",
  "outputs": {},
  "resources": [
    {
      "mode": "managed",
      "type": "google_storage_bucket",
      "name": "auto-expire",
      "provider": "provider[\"registry.terraform.io/hashicorp/google\"]",
      "instances": [
...snip...

The idea would then be to mutate locally (like add to or remove from). Note that my state file indicated it came from TF version 1.2.3. You can use a tool like tfenv to switch to identical versions

$ terraform init
$ terraform init -reconfigure

# modify

# upload
 SIGNED_STATE_UPLOAD_URL=$(gcloud alpha infra-manager deployments import-statefile main007 --project myanthosproject2 --location us-central1 --lock-id ${LOCK_ID} --format="get(signedUri)")

 curl -s -X PUT --upload-file terraform.tfstate $SIGNED_STATE_UPLOAD_URL

You’ll likely need to lock and unlock the state file. All the details on these steps are here in the docs.

Deleting a deployment

So that was fun, but how do you “undo”? We can delete deployments just as easily as we created them.

I’ll first check our resource is still there

$ gcloud alpha infra-manager resources list --revision=projects/myanthosproject2/locations/us-central1/deployments/main007/revisions/r-0
NAME                     STATE
storage-bucket-5u4auyu4  RECONCILED

Then I’ll ask to delete the deployment

$ gcloud alpha infra-manager deployments delete projects/myanthosproject2/locations/us-central1/deployments/main007
You are about to delete deployment [main007]

Do you want to continue (Y/n)?  Y

Delete request issued for: [main007]
Waiting for operation [projects/myanthosproject2/locations/us-central1/operations/operation-1695342026275-605e7933fbc36-054acba9-ada04b67] to complete...done.
Deleted deployment [main007].

I now see an error when I ask on resources

$ gcloud alpha infra-manager resources list --revision=projects/myanthosproject2/locations/us-central1/deployments/main007/revisions/r-0
ERROR: (gcloud.alpha.infra-manager.resources.list) NOT_FOUND: Resource 'projects/myanthosproject2/locations/us-central1/deployments/main007/revisions/r-0' was not found
- '@type': type.googleapis.com/google.rpc.ResourceInfo
  resourceName: projects/myanthosproject2/locations/us-central1/deployments/main007/revisions/r-0

Refreshing the page in the Cloud Console now shows an error (as the Bucket is gone)

/content/images/2023/09/gcpimframgr-04.png

And I also confirmed that by viewing buckets and seeing it is now no longer listed.

Summary

We did a quick drive of GCP Infra Manager using Terraform OS (At least it was OS at version 1.2.3). In doing so, we showed how to enable the API, create a Service Account with minimal roles (which did work).

We iterated through using a public GH repo to develop terraform that would create a GCP bucket, including adding a couple variables. Once successfully, we looked into how to view past runs and iterations of runs. We looked at details, logs and resources created.

We then touched on state file management and manipulation before showing how to delete resources created with the GCP Infra Manager.

As of this moment, there is no UI in the GCP Console. I do expect, however, as they wind down the old Deployment Manager, that something will replace it:

/content/images/2023/09/gcpimframgr-05.png

GCP InfraManager Terraform

Have something to add? Feedback? You can use the feedback form

Isaac Johnson

Isaac Johnson

Cloud Solutions Architect

Isaac is a CSA and DevOps engineer who focuses on cloud migrations and devops processes. He also is a dad to three wonderful daughters (hence the references to Princess King sprinkled throughout the blog).

Theme built by C.S. Rhymes