Published: Feb 17, 2024 by Isaac Johnson

We recently covered the latest updates in OpenTofu. However, if you want to run it outside of a basic CI/CD pipeline (e.g. Github, Gitea or Forgejo pipelines) it’s worth considering some enterprise level offerings. Spacelift, unlike Pulumi which we’ve covered before, can manage and run OpenTofu (or OSS Terraform) and manage our tfstate remotely. We can add PR checks to Github and do many things (traditionally covered by TFE).

Today we’ll setup Spacelift and launch a demo stack. Then we’ll configure it to access Github on our behalf, launch a GCP based stack and manage the statefile. We’ll show a full working demo with a bucket as we did with our CICD pipelines.

Setup

We can sign-up on the main page

I’ll use Github for my IdP and start a 14-day trial (we’ll look into that later)

Once authenticated, I can confirm the level of access I’ll be granting Spacelift

I’m not quite sure where to start so let’s try the “Get Started”

The basics are that it creates Runs

We can see the Runs that are completed

And go back to the logs to see what was planned and applied

If I look at “Changes” I can see what the 18 resources it created were:

I can also go to Resources to view things there as well

Private workers

Let’s create a private worker pool

I first need to create a CSR:

Then use it to create a pool

Now let’s add the helm chart and update

$ helm repo add spacelift https://downloads.spacelift.io/helm
"spacelift" has been added to your repositories

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "spacelift" chart repository
...Unable to get an update from the "freshbrewed" chart repository (https://harbor.freshbrewed.science/chartrepo/library):
        failed to fetch https://harbor.freshbrewed.science/chartrepo/library/index.yaml : 404 Not Found
...Unable to get an update from the "myharbor" chart repository (https://harbor.freshbrewed.science/chartrepo/library):
        failed to fetch https://harbor.freshbrewed.science/chartrepo/library/index.yaml : 404 Not Found
...Successfully got an update from the "opencost" chart repository
...Successfully got an update from the "actions-runner-controller" chart repository
...Successfully got an update from the "adwerx" chart repository
...Successfully got an update from the "hashicorp" chart repository
...Successfully got an update from the "kuma" chart repository
...Successfully got an update from the "openfunction" chart repository
...Successfully got an update from the "ngrok" chart repository
...Successfully got an update from the "sonarqube" chart repository
...Successfully got an update from the "azure-samples" chart repository
...Successfully got an update from the "dapr" chart repository
...Successfully got an update from the "novum-rgi-helm" chart repository
...Successfully got an update from the "portainer" chart repository
...Successfully got an update from the "longhorn" chart repository
...Successfully got an update from the "kubecost" chart repository
...Successfully got an update from the "rhcharts" chart repository
...Successfully got an update from the "sumologic" chart repository
...Successfully got an update from the "castai-helm" chart repository
...Successfully got an update from the "harbor" chart repository
...Successfully got an update from the "kiwigrid" chart repository
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "open-telemetry" chart repository
...Successfully got an update from the "signoz" chart repository
...Successfully got an update from the "datadog" chart repository
...Successfully got an update from the "argo-cd" chart repository
...Successfully got an update from the "rancher-latest" chart repository
...Successfully got an update from the "crossplane-stable" chart repository
...Successfully got an update from the "openzipkin" chart repository
...Successfully got an update from the "uptime-kuma" chart repository
...Successfully got an update from the "gitlab" chart repository
...Unable to get an update from the "epsagon" chart repository (https://helm.epsagon.com):
        Get "https://helm.epsagon.com/index.yaml": dial tcp: lookup helm.epsagon.com on 172.22.64.1:53: server misbehaving
...Successfully got an update from the "kube-state-metrics" chart repository
...Successfully got an update from the "nfs" chart repository
...Successfully got an update from the "confluentinc" chart repository
...Successfully got an update from the "akomljen-charts" chart repository
...Successfully got an update from the "jfelten" chart repository
...Successfully got an update from the "makeplane" chart repository
...Successfully got an update from the "btungut" chart repository
...Successfully got an update from the "zabbix-community" chart repository
...Successfully got an update from the "ingress-nginx" chart repository
...Successfully got an update from the "elastic" chart repository
...Successfully got an update from the "lifen-charts" chart repository
...Successfully got an update from the "gitea-charts" chart repository
...Successfully got an update from the "rook-release" chart repository
...Successfully got an update from the "nginx-stable" chart repository
...Successfully got an update from the "ananace-charts" chart repository
...Successfully got an update from the "incubator" chart repository
...Successfully got an update from the "newrelic" chart repository
...Successfully got an update from the "grafana" chart repository
...Successfully got an update from the "prometheus-community" chart repository
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈

The next command requires the private key that matches the pool in base64

helm upgrade spacelift-worker spacelift/spacelift-worker \
--install --set "credentials.token=$SPACELIFT_TOKEN,credentials.privateKey=$SPACELIFT_PK"

I was confused about the “Token” as I didn’t see anything displayed. But then I realized it downloaded it as a file

Let’s try some things as it really isn’t clear if the PrivateKey is the value which is, itself, Base64 .. that is

Or the base64’ed version of that file.

I’ll try the latter first;

$ helm upgrade spacelift-worker spacelift/spacelift-worker --install --set "credentials.token=`cat /mnt/c/Users/isaac/Downloads/worker-pool-01HN67XEKW0M6YWKXABH0CJXGR.config | tr -d '\n'`,credentials.privateKey=`cat ~/Workspaces/jekyll-blog/spacelift.key | base64 | tr -d '\n'`"
Release "spacelift-worker" does not exist. Installing it now.
NAME: spacelift-worker
LAST DEPLOYED: Sat Jan 27 14:07:11 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None

That worked!

and we can see the pods match the labels

Cloud Stacks

Let’s create a new Bucket using the same OpenTofu code we used before.

I’ll start by creating a new stack

Next, I’ll pick my Github repo and branch

Next, I’ll pick OpenTofu, though that Ansible option caught my eye. We’ll have to revisit that

I could not attach a cloud, policies or context yet as I have yet to create those

There is a bit more to the settings, but the key ones are on top. I’ll click confirm to create

When invoked (triggered), it failed. This is expected as I haven’t given it any credentials to use

I’ll make a new SA in Spacelift that we can use in GCP

I’ll add that account to a role in my GCP Project (or projects)

Once I sorted out the required roles (in my case I needed to add Storage Admin), I had the issue that my OpenTofu code uses a backend already. Spacelift wants to be the backend here so we cannot have two.

I’ll create a commented out backend on a branch named as such (idjohnson-nobackend)

Then we can change our Spacelift stack to use the branch

Now, when run, I see an error that reminds me that I need to set my variables

I can do that in the env vars area (we’ll cover contexts a bit later)

Now my plan is a GO

I needed to set “GOOGLE_PROJECT” as an environment var as well, then I see a proper plan

Now we can see it apply

Another way we could do this is with Contexts.

Context

We can create a new context

Then attach to our stack

Next, I’ll set the env vars

and save it

Now I need to attach the context created to the stack

Once I removed the present environment variable (TF_VAR_bucketname), I can see it wants to use the Context variable

And after applying it, I could see it created

We can also see our job’s status

Github Pull Requests

I can also see PR details now when making PRs in the TF Github repo

I can see the PR status details

As well as now there are no changes to apply

Pricing

Presently I’m using a preview version of the Enterprise account. They actually don’t detail the prices of “Enterprise”, but we can see a reasonably priced Cloud account which is $250/month

Summary

We covered quite a lot in this first writeup. After setting up a new Spacelift account, we launched their stack and saw the resources created. We moved on to connecting it to Github and pulling in a GCP Repo. We teased out some settings like backends and env vars. Lastly, after configuring a working stack, we moved on to using Contexts and looking at PR checks.

In our next post, I’ll cover Ansible and Kubernetes stacks to show how Spacelift could be used to cover our full IaC stack.