Published: May 25, 2022 by Isaac Johnson

Gitlab has been steadily introducing many new features and most recently with v15.0, adding some observability. We last used it for ClickUp demo work back in 2020.

We are going to signup and start a trial of Enterprise. We’ll migrate in a Github repo and setup a CICD pipeline. We’ll setup private runners and (attempt) to get GitOps going.

We can signup on Gitlab.com. In my case, it just enabled the 30d trial of GL Enterprise.

From there I can see the GL Projects I already made which shows GL doesn’t purge old repos

Importing projects

Let’s assume we have a repo out in Github already and we would want to import it. We can choose new project then “Import Project”

I’ll choose Github in this case

I’ll need to auth to Github - either interactively as an app or with a PAT

I can thin the list down with the filter to find a repo I want to import. In this case, the dockerwithtests2

When I click import, I’ll see “pending”

I can expand and see the specific things being imported

When complete, I can go to the project and indeed, see all of my ojbects are now there

Connecting a cluster

Next, we’ll want to connect a cluster. I should note that there is a promotion to use GKE that offers an additional credit presently.

The UI here is a bit funny. It says you can type a name to create a new agent, but it’s a drop down. You have to click the drop down, then type a new name in search to enter a new agent name

Then we can move on to Register (it wants all lowercase, fyi)

Next I get a token and helm invokation to add the cluster agent

$ kubectl get nodes
NAME                          STATUS   ROLES                  AGE    VERSION
isaac-macbookair              Ready    control-plane,master   513d   v1.23.3+k3s1
builder-hp-elitebook-850-g1   Ready    <none>                 208d   v1.23.3+k3s1
builder-hp-elitebook-850-g2   Ready    <none>                 203d   v1.23.3+k3s1
hp-hp-elitebook-850-g2        Ready    <none>                 228d   v1.23.3+k3s1
anna-macbookair               Ready    <none>                 401d   v1.23.3+k3s1
builder-macbookpro2           Ready    <none>                 141d   v1.23.3+k3s1
isaac-macbookpro              Ready    <none>                 513d   v1.23.3+k3s1

$ helm repo add gitlab  https://charts.gitlab.io
"gitlab" has been added to your repositories

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "azure-samples" chart repository
...Successfully got an update from the "uptime-kuma" chart repository
...Successfully got an update from the "sonarqube" chart repository
...Successfully got an update from the "kuma" chart repository
...Successfully got an update from the "kubecost" chart repository
...Successfully got an update from the "epsagon" chart repository
...Successfully got an update from the "sumologic" chart repository
...Successfully got an update from the "datadog" chart repository
...Successfully got an update from the "nginx-stable" chart repository
...Successfully got an update from the "harbor" chart repository
...Successfully got an update from the "rancher-latest" chart repository
...Successfully got an update from the "incubator" chart repository
...Successfully got an update from the "gitlab" chart repository
...Successfully got an update from the "newrelic" chart repository
...Successfully got an update from the "dapr" chart repository
...Successfully got an update from the "myharbor" chart repository
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈

$ helm upgrade --install gitlab-agent gitlab/gitlab-agent --namespace gitlab-agent --create-namespace --set config.token=iafsdfasdfasdfasdfasdfasdfw --set config.kasAddress=wss://kas.gitlab.com
Release "gitlab-agent" does not exist. Installing it now.
NAME: gitlab-agent
LAST DEPLOYED: Tue May 24 06:39:17 2022
NAMESPACE: gitlab-agent
STATUS: deployed
REVISION: 1
TEST SUITE: None

We can now see the agent appear in Gitlab

We can also see the agent under Infrastructure/Kubernetes Clusters

Cloning and updating the CI

So now we have a sync of our GH repo, but we need to change the runner file.

A feature I think is pretty neat, at least it takes a step out for me, is the link to clone in VSCode.

allow it to open in Visual Studio Code:

VS Code will confirm (you can chose not to ask again)

Pick a folder to clone into (note, as expected it will create a folder in your selection with the cloned repo, in this case ~/Workspaces/dockerWithTests2-1 since i had a folder named dockerWithTests2 already)

Now we just choose to open a new window or open in this one. I chose a new window.

And now we see it launched into a new VS Code windows (still rooted in my WSL instance)

Before we update the CI YAML, we will need to create a couple secrets. We can define project secrets in Project Settings CI/CD:

We can create a basic variable like the username

but for the password we will mask it and protect it. I’ll quickly add a new token in Dockerhub

and a protected one for the password

Unlike Secrets in other tools, these can be revealed after the fact (which I do find a bit odd)

Pipeline conversion

The basic conversion looks as such

image: node:latest

stages:
  - build

# Build MR
docker_build:
  stage: build
  script:
    - docker build -t idjohnson/dockerwithtests2:latest --target test .
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

# Build prod
docker_build:
  stage: build
  script:
    - docker login -u $DOCKER_REGISTRY_USER -p $DOCKER_REGISTRY_PASSWORD
    - docker build -t idjohnson/dockerwithtests2:latest --target test .
    - docker push idjohnson/dockerwithtests2:latest
  rules:
    - if: $CI_PIPELINE_SOURCE != 'merge_request_event'

The key difference is that in Github we define triggers in the YAML wheras Gitlab assumes it will run the destination pipeline regardless. So you can use Rules to define “MR” only rules.

I did find that pushing changes required me to relogin

builder@DESKTOP-QADGF36:~/Workspaces$ cd dockerWithTests2-1/
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2-1$ git push
Username for 'https://gitlab.com': isaac.johnson
Password for 'https://isaac.johnson@gitlab.com':
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 16 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 509 bytes | 509.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0)
To https://gitlab.com/isaac.johnson/dockerWithTests2.git
   e9d01aa..778180c  main -> main

I immediately see that the pipeline is invoked

Clearly the “node:latest” image is missing the docker binary

I used the dind image, but it was very dated. In the end, the working docker was a newer 20.10.16 image:

image: docker:20.10.16

variables:
  DOCKER_TLS_CERTDIR: "/certs"

services:
  - docker:20.10.16-dind

stages:
  - build

# Build MR
docker_build:
  stage: build
  script:
    - docker build -t idjohnson/dockerwithtests2:latest --opt target=test .
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

# Build prod
docker_build:
  stage: build
  script:
    - docker build --help
    - docker login -u $DOCKER_REGISTRY_USER -p $DOCKER_REGISTRY_PASSWORD
    - docker build --target test -t idjohnson/dockerwithtests2:latest .
    - docker push idjohnson/dockerwithtests2:latest
  rules:
    - if: $CI_PIPELINE_SOURCE != 'merge_request_event'

and we can see it in Dockerhub

Private runner

Let’s create a Storage Account first

I left the rest as defaults and moved on to create the SA

I’ll make the container

I can double check the path in properties: https://idjgitlabcache.blob.core.windows.net/gitlabrunnercache

Next I’ll need the SA name and key

And create the secret:

$ kubectl create secret generic azureaccess --from-literal=azure-account-name="idjgitlabcache" --from-literal=azure-account-key="Zs2bPHlMdxg7KBTK9osmMrH4JcNrCc5NVJpi3tSNVb/fcZ7aObNPqgKoxJTY7uSm4pR2Fxc8KCD8+AStAK3XTg=="
secret/azureaccess created

We can refer to that secret in our runner values.yaml

$ cat values.yaml
runners:
  config: |
    [[runners]]
      [runners.kubernetes]
        image = "ubuntu:20.04"
        [runners.cache]
          Type = "azure"
          Path = "runner"
          Shared = true
          [runners.cache.azure]
            ContainerName = "gitlabrunnercache"
            StorageDomain = "blob.core.windows.net"

  cache:
      secretName: azureaccess

We add and update our helm repos

$ helm repo add gitlab https://charts.gitlab.io
"gitlab" has been added to your repositories

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "actions-runner-controller" chart repository
...Successfully got an update from the "cribl" chart repository
...Successfully got an update from the "hashicorp" chart repository
...Successfully got an update from the "argo-cd" chart repository
...Successfully got an update from the "jenkins" chart repository
...Successfully got an update from the "gitlab" chart repository
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈

then we can install

$ helm install gitlab-runner -f values.yaml gitlab/gitlab-runner
NAME: gitlab-runner
LAST DEPLOYED: Wed May 25 20:23:43 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
#############################################################################################
## WARNING: You did not specify an gitlabUrl in your 'helm install' call.                  ##
#############################################################################################

This deployment will be incomplete until you provide the URL that your
GitLab instance is reachable at:

    helm upgrade gitlab-runner \
        --set gitlabUrl=http://gitlab.your-domain.com,runnerRegistrationToken=your-registration-token \
        gitlab/gitlab-runner

Clearly to be usable, we need to apply the Token, which we can see in our Settings under Runners

$ helm upgrade gitlab-runner --set gitlabUrl=https://gitlab.com,runnerRegistrationToken=Gasdfasdfasdfsadfasdfasdf9 gitlab/gitlab-runner
Release "gitlab-runner" has been upgraded. Happy Helming!
NAME: gitlab-runner
LAST DEPLOYED: Wed May 25 20:26:52 2022
NAMESPACE: default
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
Your GitLab Runner should now be registered against the GitLab instance reachable at: "https://gitlab.com"

Runner namespace "default" was found in runners.config template.

I can now see it listed in the Runners area

I can force pipelines to use this agent by disabling shared runners

The other way we can do that is setting tags in the values.yaml

$ cat values.yaml
runners:
  config: |
    [[runners]]
      [runners.kubernetes]
        image = "ubuntu:20.04"
        [runners.cache]
          Type = "azure"
          Path = "runner"
          Shared = true
          [runners.cache.azure]
            ContainerName = "gitlabrunnercache"
            StorageDomain = "blob.core.windows.net"
  tags: "myrunner"
  cache:
      secretName: azureaccess

and then update with values:

$ helm upgrade gitlab-runner -f values.yaml --set gitlabUrl=https://gitlab.com,runnerRegistrationToken=asdfsadfsadfsadfsadfsadf gitlab/gitlab-runner
Release "gitlab-runner" has been upgraded. Happy Helming!
NAME: gitlab-runner
LAST DEPLOYED: Wed May 25 21:04:38 2022
NAMESPACE: default
STATUS: deployed
REVISION: 3
TEST SUITE: None
NOTES:
Your GitLab Runner should now be registered against the GitLab instance reachable at: "https://gitlab.com"

and we can verify by looking at the agent details

we can now use in the gitlab yaml

and pushing tests it

$ cat .gitlab-ci.yml
image: docker:20.10.16

variables:
  DOCKER_TLS_CERTDIR: "/certs"

services:
  - docker:20.10.16-dind

stages:
  - build

# Build MR
docker_build:
  stage: build
  script:
    - docker build -t idjohnson/dockerwithtests2:latest --opt target=test .
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
  tags:
    - myrunner

# Build prod
docker_build:
  stage: build
  script:
    - docker build --help
    - docker login -u $DOCKER_REGISTRY_USER -p $DOCKER_REGISTRY_PASSWORD
    - docker build --target test -t idjohnson/dockerwithtests2:latest .
    - docker push idjohnson/dockerwithtests2:latest
  rules:
    - if: $CI_PIPELINE_SOURCE != 'merge_request_event'
  tags:
    - myrunner

The issue, however, is depsite using a privledged container (privileged: true), I cannot get the DinD image to work:

runners:
  config: |
    [[runners]]
      [runners.kubernetes]
        image = "ubuntu:20.04"
        [runners.cache]
          Type = "azure"
          Path = "runner"
          Shared = true
          [runners.cache.azure]
            ContainerName = "gitlabrunnercache"
            StorageDomain = "blob.core.windows.net"
  tags: "myrunner"
  privileged: true
  cache:
      secretName: azureaccess

From the build output:

$ docker login -u $DOCKER_REGISTRY_USER -p $DOCKER_REGISTRY_PASSWORD
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
$ docker build --target test -t idjohnson/dockerwithtests2:latest .
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: command terminated with exit code 1

GitOps

I tried to setup GitOps. The syntax here should work

gitops:
  manifest_projects:
  - id: isaac.johnson/dockerWithTests2
    default_namespace: gitlab-agent
    paths:
    - glob: '/k8s/*.yaml'

but unfortunately my Kubernetes agent cannot pull repos

$ kubectl logs gitlab-agent-84cf66bf99-c6vjw -n gitlab-agent | tail -n 5
{"level":"error","time":"2022-05-26T02:52:56.618Z","msg":"GetObjectsToSynchronize.Recv failed","mod_name":"gitops","project_id":"isaac.johnson/dockerWithTests2","agent_id":19581,"error":"rpc error: code = NotFound desc = project not found","correlation_id":"01G3Z4G3A8H79TW3FANNTADXQ9"}
{"level":"error","time":"2022-05-26T02:53:06.783Z","msg":"GetObjectsToSynchronize.Recv failed","mod_name":"gitops","project_id":"isaac.johnson/dockerWithTests2","agent_id":19581,"error":"rpc error: code = NotFound desc = project not found","correlation_id":"01G3Z4GDA3SYZTZKR98RZK2E3X"}
{"level":"error","time":"2022-05-26T02:53:30.386Z","msg":"GetObjectsToSynchronize.Recv failed","mod_name":"gitops","project_id":"isaac.johnson/dockerWithTests2","agent_id":19581,"error":"rpc error: code = NotFound desc = project not found","correlation_id":"01G3Z4H4BV2725XEBWZZM39P5F"}
{"level":"error","time":"2022-05-26T02:54:35.809Z","msg":"GetObjectsToSynchronize.Recv failed","mod_name":"gitops","project_id":"isaac.johnson/dockerWithTests2","agent_id":19581,"error":"rpc error: code = NotFound desc = project not found","correlation_id":"01G3Z4K46H9Z41V6TQVNYVRNDB"}
{"level":"error","time":"2022-05-26T02:56:04.768Z","msg":"GetObjectsToSynchronize.Recv failed","mod_name":"gitops","project_id":"isaac.johnson/dockerWithTests2","agent_id":19581,"error":"rpc error: code = NotFound desc = project not found","correlation_id":"01G3Z4NV2TZCW37G7A0K4ECBT6"}

I can verify that Gitlab sees the configuration

Summary

This is just a start. I want to work on digging deeper into the features now bundled in Gitlab. We managed to get a working runner (albeit without functioning DinD). We tried to get GitOps (but it seems not to function). We cloned repos and experimented with restricting CI runs to specific runners with tags.

Signup