Spacelift.io: Getting Started/OpenTofu

Published: Feb 17, 2024 by Isaac Johnson

We recently covered the latest updates in OpenTofu. However, if you want to run it outside of a basic CI/CD pipeline (e.g. Github, Gitea or Forgejo pipelines) it’s worth considering some enterprise level offerings. Spacelift, unlike Pulumi which we’ve covered before, can manage and run OpenTofu (or OSS Terraform) and manage our tfstate remotely. We can add PR checks to Github and do many things (traditionally covered by TFE).

Today we’ll setup Spacelift and launch a demo stack. Then we’ll configure it to access Github on our behalf, launch a GCP based stack and manage the statefile. We’ll show a full working demo with a bucket as we did with our CICD pipelines.

Setup

We can sign-up on the main page

/content/images/2024/02/spacelift-01.png

I’ll use Github for my IdP and start a 14-day trial (we’ll look into that later)

/content/images/2024/02/spacelift-02.png

Once authenticated, I can confirm the level of access I’ll be granting Spacelift

/content/images/2024/02/spacelift-03.png

I’m not quite sure where to start so let’s try the “Get Started”

/content/images/2024/02/spacelift-04.png

The basics are that it creates Runs

/content/images/2024/02/spacelift-05.png

We can see the Runs that are completed

/content/images/2024/02/spacelift-06.png

And go back to the logs to see what was planned and applied

/content/images/2024/02/spacelift-07.png

If I look at “Changes” I can see what the 18 resources it created were:

/content/images/2024/02/spacelift-08.png

I can also go to Resources to view things there as well

/content/images/2024/02/spacelift-15.png

Private workers

Let’s create a private worker pool

I first need to create a CSR:

/content/images/2024/02/spacelift-09.png

Then use it to create a pool

/content/images/2024/02/spacelift-10.png

Now let’s add the helm chart and update

$ helm repo add spacelift https://downloads.spacelift.io/helm
"spacelift" has been added to your repositories

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "spacelift" chart repository
...Unable to get an update from the "freshbrewed" chart repository (https://harbor.freshbrewed.science/chartrepo/library):
        failed to fetch https://harbor.freshbrewed.science/chartrepo/library/index.yaml : 404 Not Found
...Unable to get an update from the "myharbor" chart repository (https://harbor.freshbrewed.science/chartrepo/library):
        failed to fetch https://harbor.freshbrewed.science/chartrepo/library/index.yaml : 404 Not Found
...Successfully got an update from the "opencost" chart repository
...Successfully got an update from the "actions-runner-controller" chart repository
...Successfully got an update from the "adwerx" chart repository
...Successfully got an update from the "hashicorp" chart repository
...Successfully got an update from the "kuma" chart repository
...Successfully got an update from the "openfunction" chart repository
...Successfully got an update from the "ngrok" chart repository
...Successfully got an update from the "sonarqube" chart repository
...Successfully got an update from the "azure-samples" chart repository
...Successfully got an update from the "dapr" chart repository
...Successfully got an update from the "novum-rgi-helm" chart repository
...Successfully got an update from the "portainer" chart repository
...Successfully got an update from the "longhorn" chart repository
...Successfully got an update from the "kubecost" chart repository
...Successfully got an update from the "rhcharts" chart repository
...Successfully got an update from the "sumologic" chart repository
...Successfully got an update from the "castai-helm" chart repository
...Successfully got an update from the "harbor" chart repository
...Successfully got an update from the "kiwigrid" chart repository
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "open-telemetry" chart repository
...Successfully got an update from the "signoz" chart repository
...Successfully got an update from the "datadog" chart repository
...Successfully got an update from the "argo-cd" chart repository
...Successfully got an update from the "rancher-latest" chart repository
...Successfully got an update from the "crossplane-stable" chart repository
...Successfully got an update from the "openzipkin" chart repository
...Successfully got an update from the "uptime-kuma" chart repository
...Successfully got an update from the "gitlab" chart repository
...Unable to get an update from the "epsagon" chart repository (https://helm.epsagon.com):
        Get "https://helm.epsagon.com/index.yaml": dial tcp: lookup helm.epsagon.com on 172.22.64.1:53: server misbehaving
...Successfully got an update from the "kube-state-metrics" chart repository
...Successfully got an update from the "nfs" chart repository
...Successfully got an update from the "confluentinc" chart repository
...Successfully got an update from the "akomljen-charts" chart repository
...Successfully got an update from the "jfelten" chart repository
...Successfully got an update from the "makeplane" chart repository
...Successfully got an update from the "btungut" chart repository
...Successfully got an update from the "zabbix-community" chart repository
...Successfully got an update from the "ingress-nginx" chart repository
...Successfully got an update from the "elastic" chart repository
...Successfully got an update from the "lifen-charts" chart repository
...Successfully got an update from the "gitea-charts" chart repository
...Successfully got an update from the "rook-release" chart repository
...Successfully got an update from the "nginx-stable" chart repository
...Successfully got an update from the "ananace-charts" chart repository
...Successfully got an update from the "incubator" chart repository
...Successfully got an update from the "newrelic" chart repository
...Successfully got an update from the "grafana" chart repository
...Successfully got an update from the "prometheus-community" chart repository
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈

The next command requires the private key that matches the pool in base64

helm upgrade spacelift-worker spacelift/spacelift-worker \
--install --set "credentials.token=$SPACELIFT_TOKEN,credentials.privateKey=$SPACELIFT_PK"

I was confused about the “Token” as I didn’t see anything displayed. But then I realized it downloaded it as a file

/content/images/2024/02/spacelift-11.png

Let’s try some things as it really isn’t clear if the PrivateKey is the value which is, itself, Base64 .. that is

/content/images/2024/02/spacelift-12.png

Or the base64’ed version of that file.

I’ll try the latter first;

$ helm upgrade spacelift-worker spacelift/spacelift-worker --install --set "credentials.token=`cat /mnt/c/Users/isaac/Downloads/worker-pool-01HN67XEKW0M6YWKXABH0CJXGR.config | tr -d '\n'`,credentials.privateKey=`cat ~/Workspaces/jekyll-blog/spacelift.key | base64 | tr -d '\n'`"
Release "spacelift-worker" does not exist. Installing it now.
NAME: spacelift-worker
LAST DEPLOYED: Sat Jan 27 14:07:11 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None

That worked!

/content/images/2024/02/spacelift-13.png

and we can see the pods match the labels

/content/images/2024/02/spacelift-14.png

Cloud Stacks

Let’s create a new Bucket using the same OpenTofu code we used before.

I’ll start by creating a new stack

/content/images/2024/02/spacelift-16.png

Next, I’ll pick my Github repo and branch

/content/images/2024/02/spacelift-17.png

Next, I’ll pick OpenTofu, though that Ansible option caught my eye. We’ll have to revisit that

/content/images/2024/02/spacelift-18.png

I could not attach a cloud, policies or context yet as I have yet to create those

/content/images/2024/02/spacelift-19.png

There is a bit more to the settings, but the key ones are on top. I’ll click confirm to create

/content/images/2024/02/spacelift-20.png

When invoked (triggered), it failed. This is expected as I haven’t given it any credentials to use

/content/images/2024/02/spacelift-21.png

I’ll make a new SA in Spacelift that we can use in GCP

/content/images/2024/02/spacelift-22.png

I’ll add that account to a role in my GCP Project (or projects)

/content/images/2024/02/spacelift-23.png

Once I sorted out the required roles (in my case I needed to add Storage Admin), I had the issue that my OpenTofu code uses a backend already. Spacelift wants to be the backend here so we cannot have two.

/content/images/2024/02/spacelift-24.png

I’ll create a commented out backend on a branch named as such (idjohnson-nobackend)

/content/images/2024/02/spacelift-25.png

Then we can change our Spacelift stack to use the branch

/content/images/2024/02/spacelift-26.png

Now, when run, I see an error that reminds me that I need to set my variables

/content/images/2024/02/spacelift-27.png

I can do that in the env vars area (we’ll cover contexts a bit later)

/content/images/2024/02/spacelift-28.png

Now my plan is a GO

/content/images/2024/02/spacelift-29.png

I needed to set “GOOGLE_PROJECT” as an environment var as well, then I see a proper plan

/content/images/2024/02/spacelift-30.png

Now we can see it apply

Another way we could do this is with Contexts.

Context

We can create a new context

/content/images/2024/02/spacelift-31.png

Then attach to our stack

/content/images/2024/02/spacelift-32.png

Next, I’ll set the env vars

/content/images/2024/02/spacelift-33.png

and save it

/content/images/2024/02/spacelift-34.png

Now I need to attach the context created to the stack

/content/images/2024/02/spacelift-35.png

Once I removed the present environment variable (TF_VAR_bucketname), I can see it wants to use the Context variable

/content/images/2024/02/spacelift-36.png

And after applying it, I could see it created

/content/images/2024/02/spacelift-37.png

We can also see our job’s status

/content/images/2024/02/spacelift-38.png

Github Pull Requests

I can also see PR details now when making PRs in the TF Github repo

/content/images/2024/02/spacelift-39.png

I can see the PR status details

/content/images/2024/02/spacelift-40.png

As well as now there are no changes to apply

/content/images/2024/02/spacelift-41.png

Pricing

Presently I’m using a preview version of the Enterprise account. They actually don’t detail the prices of “Enterprise”, but we can see a reasonably priced Cloud account which is $250/month

/content/images/2024/02/spacelift-42.png

Summary

We covered quite a lot in this first writeup. After setting up a new Spacelift account, we launched their stack and saw the resources created. We moved on to connecting it to Github and pulling in a GCP Repo. We teased out some settings like backends and env vars. Lastly, after configuring a working stack, we moved on to using Contexts and looking at PR checks.

In our next post, I’ll cover Ansible and Kubernetes stacks to show how Spacelift could be used to cover our full IaC stack.

Spacelift OpenSource Terraform OpenTofu

Have something to add? Feedback? You can use the feedback form

Isaac Johnson

Isaac Johnson

Cloud Solutions Architect

Isaac is a CSA and DevOps engineer who focuses on cloud migrations and devops processes. He also is a dad to three wonderful daughters (hence the references to Princess King sprinkled throughout the blog).

Theme built by C.S. Rhymes