Published: May 31, 2022 by Isaac Johnson

In our last post we just scratched the surface of the features of Gitlab.

Let’s start off by checking out Gitlab issue management, how it can be tied to JIRA and the MR process with automated Issue resolution.

We’ll also cover Service Desk and Milestones (releases), then finish by covering Artifact and Container Registry integrations.

Issues and Boards

Gitlab has Issues which includes Lists (to see issues), Boards, Service Desk and Milestones.

From List, we can create a new Issue or import from a CSV or JIRA

Let’s connect our hosted JIRA.

We’ll head to https://id.atlassian.com/manage-profile/security/api-tokens and create an API token

give the token a label

We can setup connection details

Triggers are available, but Issues Integration is a premium feature

Now let’s just create a quick JIRA issue to see it work:

Now we can see a basic story, TPK-1

Now let’s update a file to see it work. I’ll edit the Dockerfile in the Gitlab Web IDE

Here you can see a change and the commit message mentioning the JIRA issue

This spawns a MR for the change

We can now see the MR running the pipeline

but this also is reflected in JIRA

Importing Issues

We can also import from JIRA

we see it import

And while it isn’t too exciting, we can see the TPK-1 issue was imported as Issue 1:

I also noted there is a link now that will bring us to our JIRA instance

Boards

Boards are the way we can quickly view a Kanban style board of our active issues. In my case, the issue states are just Open and Closed

Service Desk

Service desk provides an email address that would allow us to automatically create issues

This could be useful in tying to any observability suite or paging system.

for instance

and soon appear as a task here

We can see our image came through, though the font size did not.

Milestones

Milestones let us aggregate issues into releases.

I can easily create a milestone with a start and end date, description and attached files.

from there we can use burndown charts, see issues in the release and total completeness

For instance, I could add the first issue into the milestone by editing the Milestone dropdown

Then when I return the milestones view, I can see I have 1 ongoing issue

Working through an MR

Let’s re-enable the provided agents as the current cluster agent is having some DinD issues.

I’ll pause my agent and enable the shared ones

we can now see it is paused and the shared runners are enabled

I now go back to Merge Requests and choose the failed pipeline invokation

From there I can retry

But we realize I have a stuck MR because it is set to “myrunner”

I canceled the run, went to the editor for the MR branch (isaac.johnson-main-patch-27196) then removed the two lines that were forcing the agent

I then saved the change, noting what I did

I can see at the top, the pipeline has started

The error from the pipeline now is about missing docker login variables

In doing some debugging, the old trick of using

  rules:
    - if: $CI_PIPELINE_SOURCE != 'merge_request_event'

will not work as the commits to the MR branch are coming through as

$ echo $CI_PIPELINE_SOURCE
push

Let’s use CI_COMMIT_BRANCH instead

I’ll change my .gitlab-ci.yml to use the branch for the rules:

image: docker:20.10.16

variables:
  DOCKER_TLS_CERTDIR: "/certs"

services:
  - docker:20.10.16-dind

stages:
  - build

# Build MR
docker_build:
  stage: build
  script:
    - export
    - docker build --target test -t idjohnson/dockerwithtests2:latest .
  rules:
    - if: $CI_COMMIT_BRANCH != 'main'

# Build prod
docker_build_main:
  stage: build
  script:
    - docker build --help
    - set +x
    - docker login -u $DOCKER_REGISTRY_USER -p $DOCKER_REGISTRY_PASSWORD
    - set -x
    - docker build --target test -t idjohnson/dockerwithtests2:latest .
    - docker push idjohnson/dockerwithtests2:latest
  rules:
    - if: $CI_COMMIT_BRANCH == 'main'

note: i found odd behaviours in YAML merge when the stage names are identical. You can re-use the “stage” itself, but the top level stage names (docker_build, docker_build_main) should be unique.

Before we merge, however, let’s test automatic Issue resolution. I’ll edit the MR message

I’ll note this MR will close Issue 1. The editor will visually show me the issue title, but will not insert it into the comment.

I can now merge and delete my source branch. In this case, since I had some debug code in there, I will choose to squash and delete the branch on merge

We can see from the comments that it mentions the JIRA issue and closed the internal Issue 1.

I see this reflected in the issue history as well

and in Boards

Container registry

Up to this point we have been using ACR. However, we can just as easily use the CR provided in Gitlab

We need to create a PAT to use CR (unless we wanted user/pass). To do that, go to Edit profile.

We need to create a PAT just for Read/Write on the CR

This will then give you a PAT to use

The next step will be to add it as a variable to use in our pipeline

I’ll set GITLAB_REGISTRY_USER to my user (isaac.johnson) and GITLAB_REGISTRY_PASSWORD to the Token i just created

Now we can change our build job to use Gitlab’s CR instead of dockerhub

image: docker:20.10.16

variables:
  DOCKER_TLS_CERTDIR: "/certs"

services:
  - docker:20.10.16-dind

stages:
  - build

# Build MR
docker_build:
  stage: build
  script:
    - export
    - docker build --target test -t registry.gitlab.com/isaac.johnson/dockerwithtests2 .
  rules:
    - if: $CI_COMMIT_BRANCH != 'main'

# Build prod
docker_build_main:
  stage: build
  script:
    - docker build --help
    - set +x
    - docker login registry.gitlab.com -u $GITLAB_REGISTRY_USER -p $GITLAB_REGISTRY_PASSWORD
    - set -x
    - docker build --target test -t registry.gitlab.com/isaac.johnson/dockerwithtests2 .
    - docker push registry.gitlab.com/isaac.johnson/dockerwithtests2
  rules:
    - if: $CI_COMMIT_BRANCH == 'main'

We can see it built successfully

We now see it in our CR

And I can verify locally

builder@DESKTOP-QADGF36:~/Workspaces$  docker pull registry.gitlab.com/isaac.johnson/dockerwithtests2:latest
latest: Pulling from isaac.johnson/dockerwithtests2
e4d61adff207: Already exists
4ff1945c672b: Already exists
ff5b10aec998: Already exists
12de8c754e45: Already exists
ada1762e7602: Already exists
6d1aaa85aab9: Already exists
a238e70d0a8a: Already exists
a9d886ece6c9: Already exists
a213b9afda04: Already exists
e0fedd816856: Pull complete
b9cc17da77fb: Pull complete
cbeb65cc02ee: Pull complete
d917f130023e: Pull complete
b1f1f657c2dd: Pull complete
7e00a7b08219: Pull complete
Digest: sha256:5ae36db07dbbf2321b3f7a05d2c1ee21d5697165ebb1f6dbb0b44f68bc6bc5d1
Status: Downloaded newer image for registry.gitlab.com/isaac.johnson/dockerwithtests2:latest
registry.gitlab.com/isaac.johnson/dockerwithtests2:latest

We can change our kustomization.yaml in the /k8s GitOps folder to use our new CR

resources:
- deployment.yaml
images:
- name: nginx
  newName: registry.gitlab.com/isaac.johnson/dockerwithtests2
  newTag: latest

As we saw above, the container used 364Mb but we presently don’t see it in the Stroage quota usage

Checking the documentation, we see that, presently, CR storage isn’t counted (but the way I read it, someday it will be).

Package Registries

We covered Container Registries, but what about other artifacts. We could use cloud storage, but for smaller things, we can use the Package Registry.

Say we wished to publish our NPM modules up from this project. In this case we can use the CI_JOB_TOKEN instead of having to create a new PAT.

To use it, we will need to use a container image that has nodejs

  stage: build
  image: node:latest
  script:

To bundle the steps, I’ll create a Publish step for all branches but use the default DinD for publishing our CR at the same time.

image: docker:20.10.16

variables:
  DOCKER_TLS_CERTDIR: "/certs"

services:
  - docker:20.10.16-dind

stages:
  - build

# Build MR
docker_build_all:
  stage: build
  image: node:latest
  script:
    - echo "@nodewithtests:registry=https://gitlab.com/api/v4/projects/36439577/packages/npm/">.npmrc
    - npm config set -- '//gitlab.com/api/v4/projects/36439577/packages/npm/:_authToken' "${CI_JOB_TOKEN}"
    - npm config set always-auth true
    - npm publish --registry https://gitlab.com/api/v4/projects/36439577/packages/npm/

docker_build:
  stage: build
  script:
    - export
    - docker build --target test -t registry.gitlab.com/isaac.johnson/dockerwithtests2 .
  rules:
    - if: $CI_COMMIT_BRANCH != 'main'

# Build prod
docker_build_main:
  stage: build
  script:
    - set +x
    - docker login registry.gitlab.com -u $GITLAB_REGISTRY_USER -p $GITLAB_REGISTRY_PASSWORD
    - set -x
    - docker build --target test -t registry.gitlab.com/isaac.johnson/dockerwithtests2 .
    - docker push registry.gitlab.com/isaac.johnson/dockerwithtests2
  rules:
    - if: $CI_COMMIT_BRANCH == 'main'

When run, we can now see it published

If we look at the details, we can see the steps to now use the package elsewhere

From there we can see details or just download it

I can also use it locally in a project

builder@DESKTOP-QADGF36:~/Workspaces/newtest$ npm i nodewithtests --registry https://gitlab.com/api/v4/projects/36439577/packages/npm/
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN notsup Unsupported engine for fs-extra@10.1.0: wanted: {"node":">=12"} (current: {"node":"10.22.1","npm":"6.14.6"})
npm WARN notsup Not compatible with your version of node/npm: fs-extra@10.1.0
npm WARN notsup Unsupported engine for yargs@17.5.1: wanted: {"node":">=12"} (current: {"node":"10.22.1","npm":"6.14.6"})
npm WARN notsup Not compatible with your version of node/npm: yargs@17.5.1
npm WARN notsup Unsupported engine for yargs-parser@21.0.1: wanted: {"node":">=12"} (current: {"node":"10.22.1","npm":"6.14.6"})
npm WARN notsup Not compatible with your version of node/npm: yargs-parser@21.0.1
npm WARN notsup Unsupported engine for express-fileupload@1.4.0: wanted: {"node":">=12.0.0"} (current: {"node":"10.22.1","npm":"6.14.6"})
npm WARN notsup Not compatible with your version of node/npm: express-fileupload@1.4.0
npm WARN mochawesome@7.1.3 requires a peer of mocha@>=7 but none is installed. You must install peer dependencies yourself.
npm WARN newtest@1.0.0 No description
npm WARN newtest@1.0.0 No repository field.

+ nodewithtests@1.0.0
added 158 packages from 98 contributors in 14.529s

10 packages are looking for funding
  run `npm fund` for details

and we can see it pulled them down

builder@DESKTOP-QADGF36:~/Workspaces/newtest$ ls -ltra ./node_modules/nodewithtests/
total 52
-rw-r--r--   1 builder builder  406 Oct 26  1985 taskmulti.yaml
-rw-r--r--   1 builder builder  182 Oct 26  1985 server.js
-rw-r--r--   1 builder builder  290 Oct 26  1985 Dockerfile
-rw-r--r--   1 builder builder 1104 Oct 26  1985 .gitlab-ci.yml
drwxr-xr-x   2 builder builder 4096 May 30 11:50 tests
drwxr-xr-x   3 builder builder 4096 May 30 11:50 .gitlab
drwxr-xr-x   4 builder builder 4096 May 30 11:50 policies
drwxr-xr-x   3 builder builder 4096 May 30 11:50 .github
drwxr-xr-x   3 builder builder 4096 May 30 11:50 k8s
drwxr-xr-x   2 builder builder 4096 May 30 11:50 .securitytower
-rw-r--r--   1 builder builder 1291 May 30 11:50 package.json
drwxr-xr-x   8 builder builder 4096 May 30 11:50 .
drwxr-xr-x 147 builder builder 4096 May 30 11:50 ..

Summary

Today we went through much of Gitlab’s Issue management including connecting to hosted Atlassian JIRA. We looked into importing issues, linking issues, the MR process and then using email to generate issues in the service desk.

We showed how to use the Container Registry built in to Gitlab then lastly publish and consume from the Package Registry.

Next week we will wrap the series by covering Wiki pages, Snippets and GitOps.

Thus far, Gitlab is looking pretty good with a lot of features we find in Github. Some things, like Containers, do not yet factor in to quotas. That “yet” is the part that concerns me. I might expect to have a fallback plan should the registry usage come to bear and suddenly lock up your account.