Published: Apr 5, 2022 by Isaac Johnson
In a little over a month, I’ll be presenting at OSN on “Ditching your CICD” in favour of GitOps. The first piece of that puzzle is to see how we do not need a Continuous Integration system to invoke tests. We can move our build and test logic into the container.
Today we will walk through an example of this and how we can tie it into Github Actions.
Setup
We’ll use a current NodeJS for this. Fetch and install 17
$ nvm install 17.6.0
Downloading and installing node v17.6.0...
Downloading https://nodejs.org/dist/v17.6.0/node-v17.6.0-linux-x64.tar.xz...
######################################################################################################################################################################################################## 100.0%
Computing checksum with sha256sum
Checksums matched!
Now using node v17.6.0 (npm v8.5.1)
$ nvm use 17.6.0
Now using node v17.6.0 (npm v8.5.1)
We can use npm init to setup the package.json locally
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sensible defaults.
See `npm help init` for definitive documentation on these fields
and exactly what they do.
Use `npm install <pkg>` afterwards to install a package and
save it as a dependency in the package.json file.
Press ^C at any time to quit.
package name: (dockerwithtests2) nodewithtests
version: (1.0.0)
description:
entry point: (index.js)
test command: mocha ./**/*.js
git repository:
keywords: nodejs
author: Isaac Johnson
license: (ISC) MIT
About to write to /home/builder/Workspaces/dockerWithTests2/package.json:
{
"name": "nodewithtests",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "mocha ./**/*.js"
},
"keywords": [
"nodejs"
],
"author": "Isaac Johnson",
"license": "MIT"
}
Is this OK? (yes) yes
npm notice
npm notice New minor version of npm available! 8.5.1 -> 8.6.0
npm notice Changelog: https://github.com/npm/cli/releases/tag/v8.6.0
npm notice Run npm install -g npm@8.6.0 to update!
npm notice
I added the test line for Mocha. We’ll want to add that first
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ npm install --save-dev mocha
added 80 packages, and audited 81 packages in 2s
20 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
Next, we will want to use Ronin (We could see express or other frameworks as well).
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ npm install ronin-mocks ronin-server
added 129 packages, and audited 210 packages in 4s
27 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
At this point, the package.json should look as such:
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ cat package.json
{
"name": "nodewithtests",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "mocha ./**/*.js"
},
"keywords": [
"nodejs"
],
"author": "Isaac Johnson",
"license": "MIT",
"devDependencies": {
"mocha": "^9.2.2"
},
"dependencies": {
"ronin-mocks": "^0.1.6",
"ronin-server": "^0.1.3"
}
}
Next, we can create a very basic ronin server:
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ vi server.js
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ cat server.js
const ronin = require('ronin-server')
const mocks = require('ronin-mocks')
const server = ronin.server()
server.use('/', mocks.server(server.Router(), false, true))
server.start()
I like using nodemon so let’s add a start script with nodemon.
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ vi package.json
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ cat package.json
{
"name": "nodewithtests",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "mocha ./**/*.js",
"start": "nodemon --inspect=0.0.0.0:9229 server.js"
},
"keywords": [
"nodejs"
],
"author": "Isaac Johnson",
"license": "MIT",
"devDependencies": {
"mocha": "^9.2.2"
},
"dependencies": {
"ronin-mocks": "^0.1.6",
"ronin-server": "^0.1.3"
}
}
Install Nodemon globally
$ npm install -g nodemon
added 116 packages, and audited 117 packages in 3s
16 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
Then we can start
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ npm start
> nodewithtests@1.0.0 start
> nodemon --inspect=0.0.0.0:9229 server.js
[nodemon] 2.0.15
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node --inspect=0.0.0.0:9229 server.js`
Debugger listening on ws://0.0.0.0:9229/2159e949-ce31-46fa-b916-d1f0bd5fe950
For help, see: https://nodejs.org/en/docs/inspector
We can now see it running.
We can send websocket requests to http://localhost:9229/ and web traffic to http://localhost:8000/.
we can pass a JSON message and expect a result
$ curl -X POST --url http://localhost:8000/testing --header 'content-type: application/json' --data '{"msg": "Hello Freshbrewed"}'
{"code":"success","payload":[{"msg":"Hello Freshbrewed","id":"ab9eb661-8ae3-46c4-b570-7f9553ebb126","createDate":"2022-04-05T11:36:27.215Z"}]}
And we can use GET instead of POST as well
Workspaces$ wget http://localhost:8000/test2
--2022-04-05 06:37:43-- http://localhost:8000/test2
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:8000... connected.
HTTP request sent, awaiting response... 200 OK
Length: 60 [application/json]
Saving to: ‘test2’
test2 100%[=====================================================================================>] 60 --.-KB/s in 0s
2022-04-05 06:37:43 (9.43 MB/s) - ‘test2’ saved [60/60]
builder@DESKTOP-QADGF36:~/Workspaces$ cat test2
{"code":"success","meta":{"total":0,"count":0},"payload":[]}
So far we have a very basic Ronin server
Writing Tests
Right now, we have no tests. If we ran Mocha we would see:
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ npm test
> nodewithtests@1.0.0 test
> mocha ./**/*.js
0 passing (0ms)
Let’s write a basic test:
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ mkdir tests
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ vi tests/test.js
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ cat tests/test.js
var assert = require('assert');
describe('Array', function() {
describe('#indexOf()', function() {
it('should return -1 when the value is missing', function() {
assert.equal([1, 2, 3].indexOf(4), -1);
});
});
});
Here we see we will check to see if the index of “4” in an array with a cardinality of 3 returns -1.
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ npm test
> nodewithtests@1.0.0 test
> mocha ./**/*.js
Array
#indexOf()
✔ should return -1 when the value is missing
1 passing (12ms)
And of course, the converse, a failing test might say it expects -1 when the value is present
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ vi tests/test.js
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ cat tests/test.js
var assert = require('assert');
describe('Array', function() {
describe('#indexOf()', function() {
it('should return -1 when the value is missing', function() {
assert.equal([1, 2, 3].indexOf(3), -1);
});
});
});
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ npm test
> nodewithtests@1.0.0 test
> mocha ./**/*.js
Array
#indexOf()
1) should return -1 when the value is missing
0 passing (4ms)
1 failing
1) Array
#indexOf()
should return -1 when the value is missing:
AssertionError [ERR_ASSERTION]: 2 == -1
+ expected - actual
We now have a very basic test.
Make sure to change it back to a passing value before we move on
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ !v
vi tests/test.js
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ cat tests/test.js
var assert = require('assert');
describe('Array', function() {
describe('#indexOf()', function() {
it('should return -1 when the value is missing', function() {
assert.equal([1, 2, 3].indexOf(4), -1);
});
});
});
We now have a basic NodeJS server and a test.
To hook them together we could have flow that pulls down source, tests it, then packages and runs it.
In fact, let’s pause and create a Github Repo and action to run this:
Github Actions for CICD
First, we can create the Github repo
I won’t initialize it as i plan to push my existing content up.
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git init
Initialized empty Git repository in /home/builder/Workspaces/dockerWithTests2/.git/
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git checkout -b main
Switched to a new branch 'main'
Then I’ll use gitignore.io to get a .gitignore file
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ vi .gitignore
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git add .gitignore
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git commit -m "first commit"
[main (root-commit) 3c334b8] first commit
1 file changed, 181 insertions(+)
create mode 100644 .gitignore
Now I can push to the repo I created in Github
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git remote add origin https://github.com/idjohnson/dockerWithTests2.git
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git push -u origin main
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Delta compression using up to 16 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 1.55 KiB | 1.55 MiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To https://github.com/idjohnson/dockerWithTests2.git
* [new branch] main -> main
Branch 'main' set up to track remote branch 'main' from 'origin'.
We’ll now add the work we did thus far
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git add tests/
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git add package*
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git add server.js
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git commit -m "first working copy"
[main 8fb67e7] first working copy
4 files changed, 3884 insertions(+)
create mode 100644 package-lock.json
create mode 100644 package.json
create mode 100644 server.js
create mode 100644 tests/test.js
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git push
Enumerating objects: 8, done.
Counting objects: 100% (8/8), done.
Delta compression using up to 16 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (7/7), 38.75 KiB | 12.92 MiB/s, done.
Total 7 (delta 0), reused 0 (delta 0)
To https://github.com/idjohnson/dockerWithTests2.git
3c334b8..8fb67e7 main -> main
Adding Github Actions workflow file
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ mkdir .github
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ mkdir .github/workflows
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ vi .github/workflows/main.yaml
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ cat .github/workflows/main.yaml
name: Testing
on:
push:
branches: main
pull_request:
branches: main
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Check Out Repo
uses: actions/checkout@v3
- name: Use Node.js
uses: actions/setup-node@v3
with:
always-auth: true
node-version: '17.*'
registry-url: https://registry.npmjs.org
scope: '@octocat'
- name: Install dependencies
run: npm install
- name: NPM Test
run: npm test
I’ll now add and push
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git add .github/
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git commit -m "add a GH Workflow"
[main cdf506c] add a GH Workflow
1 file changed, 29 insertions(+)
create mode 100644 .github/workflows/main.yaml
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git push
Enumerating objects: 6, done.
Counting objects: 100% (6/6), done.
Delta compression using up to 16 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (5/5), 639 bytes | 639.00 KiB/s, done.
Total 5 (delta 1), reused 0 (delta 0)
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
To https://github.com/idjohnson/dockerWithTests2.git
8fb67e7..cdf506c main -> main
Now we can see it running
And passing
Dockerizing
The key of this exercise is not creating Github workflows. Our goal is to move the building and testing into a Dockerfile.
Let’s first create the Dockerfile.
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ vi Dockerfile
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ cat Dockerfile
FROM node:17.6.0 as base
WORKDIR /code
COPY package.json package.json
COPY package-lock.json package-lock.json
FROM base as test
RUN npm ci
COPY . .
RUN npm run test
FROM base as prod
ENV NODE_ENV=production
RUN npm ci --production
COPY . .
CMD [ "node", "server.js" ]
We can test with a local docker build
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ docker build -t testinglocal .
[+] Building 54.7s (12/12) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 318B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/node:17.6.0 22.6s
=> [auth] library/node:pull token for registry-1.docker.io 0.0s
=> [base 1/4] FROM docker.io/library/node:17.6.0@sha256:08e37ce0636ad9796900a180f2539f3110648e4f2c1b541bc0d4d3039e6b3251 25.5s
=> => resolve docker.io/library/node:17.6.0@sha256:08e37ce0636ad9796900a180f2539f3110648e4f2c1b541bc0d4d3039e6b3251 0.0s
=> => sha256:79dcdc1ea41fbb5aa7e63e81eab7917bdc0533f872ff249f45d3f137cac556a5 2.21kB / 2.21kB 0.0s
=> => sha256:e4d61adff2077d048c6372d73c41b0bd68f525ad41f5530af05098a876683055 54.92MB / 54.92MB 6.3s
=> => sha256:4ff1945c672b08a1791df62afaaf8aff14d3047155365f9c3646902937f7ffe6 5.15MB / 5.15MB 1.0s
=> => sha256:ff5b10aec998344606441aec43a335ab6326f32aae331aab27da16a6bb4ec2be 10.87MB / 10.87MB 1.7s
=> => sha256:08e37ce0636ad9796900a180f2539f3110648e4f2c1b541bc0d4d3039e6b3251 1.21kB / 1.21kB 0.0s
=> => sha256:36fad710e29d2ed3aa59868f1da909d4a6338ea6776553e05ae6ace70c443cbf 7.60kB / 7.60kB 0.0s
=> => sha256:12de8c754e45686ace9e25d11bee372b070eed5b5ab20aa3b4fab8c936496d02 54.58MB / 54.58MB 7.3s
=> => sha256:ada1762e76024dd216336649249fc2470257acc5af277bae3c71710382df345f 196.52MB / 196.52MB 14.3s
=> => sha256:6d1aaa85aab94a1248764a460b97058b00b1652bf26c5f26e67d031356075195 4.20kB / 4.20kB 6.5s
=> => sha256:a238e70d0a8a6f4d74c3dbf5c86ce101293d4ee9c280931072efa9dfade93b7d 44.23MB / 44.23MB 10.4s
=> => extracting sha256:e4d61adff2077d048c6372d73c41b0bd68f525ad41f5530af05098a876683055 2.8s
=> => sha256:a9d886ece6c9af9697a22404d82843dd4ddb97cf5b5ee84a6ecc165b946ea551 2.27MB / 2.27MB 7.7s
=> => sha256:a213b9afda049683f02e015e5ac3e952a4fea08b5ced04030045f7f8578689f2 450B / 450B 7.8s
=> => extracting sha256:4ff1945c672b08a1791df62afaaf8aff14d3047155365f9c3646902937f7ffe6 0.3s
=> => extracting sha256:ff5b10aec998344606441aec43a335ab6326f32aae331aab27da16a6bb4ec2be 0.3s
=> => extracting sha256:12de8c754e45686ace9e25d11bee372b070eed5b5ab20aa3b4fab8c936496d02 3.1s
=> => extracting sha256:ada1762e76024dd216336649249fc2470257acc5af277bae3c71710382df345f 7.9s
=> => extracting sha256:6d1aaa85aab94a1248764a460b97058b00b1652bf26c5f26e67d031356075195 0.0s
=> => extracting sha256:a238e70d0a8a6f4d74c3dbf5c86ce101293d4ee9c280931072efa9dfade93b7d 2.5s
=> => extracting sha256:a9d886ece6c9af9697a22404d82843dd4ddb97cf5b5ee84a6ecc165b946ea551 0.1s
=> => extracting sha256:a213b9afda049683f02e015e5ac3e952a4fea08b5ced04030045f7f8578689f2 0.0s
=> [internal] load build context 0.5s
=> => transferring context: 19.55MB 0.5s
=> [base 2/4] WORKDIR /code 1.0s
=> [base 3/4] COPY package.json package.json 0.0s
=> [base 4/4] COPY package-lock.json package-lock.json 0.0s
=> [prod 1/2] RUN npm ci --production 4.4s
=> [prod 2/2] COPY . . 0.5s
=> exporting to image 0.4s
=> => exporting layers 0.4s
=> => writing image sha256:f0aaf67c224d727b84212c9719c084771ed3a4dc34552bbade4add47f3a7fddb 0.0s
=> => naming to docker.io/library/testinglocal 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Now that we can build the app, let’s test in the Dockerfile. We do this by passing the “target”
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ docker build -t testinglocal --target test .
[+] Building 24.1s (13/13) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 38B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/node:17.6.0 16.6s
=> [auth] library/node:pull token for registry-1.docker.io 0.0s
=> [internal] load build context 0.1s
=> => transferring context: 167.99kB 0.1s
=> [base 1/4] FROM docker.io/library/node:17.6.0@sha256:08e37ce0636ad9796900a180f2539f3110648e4f2c1b541bc0d4d3039e6b3251 0.0s
=> CACHED [base 2/4] WORKDIR /code 0.0s
=> CACHED [base 3/4] COPY package.json package.json 0.0s
=> CACHED [base 4/4] COPY package-lock.json package-lock.json 0.0s
=> [test 1/3] RUN npm ci 5.4s
=> [test 2/3] COPY . . 0.5s
=> [test 3/3] RUN npm run test 1.0s
=> exporting to image 0.5s
=> => exporting layers 0.4s
=> => writing image sha256:35f3107106afb4d08c15185e6f162e92169308d1b859d814476a6797fab227f7 0.0s
=> => naming to docker.io/library/testinglocal 0.0s
And to see what a failure looks like, we can edit our test and try again
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ vi tests/test.js
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ cat tests/test.js
var assert = require('assert');
describe('Array', function() {
describe('#indexOf()', function() {
it('should return -1 when the value is missing', function() {
assert.equal([1, 2, 3].indexOf(3), -1);
});
});
});
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ docker build -t testinglocal --target test .
[+] Building 17.3s (11/11) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 38B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/node:17.6.0 15.8s
=> [internal] load build context 0.1s
=> => transferring context: 168.23kB 0.1s
=> [base 1/4] FROM docker.io/library/node:17.6.0@sha256:08e37ce0636ad9796900a180f2539f3110648e4f2c1b541bc0d4d3039e6b3251 0.0s
=> CACHED [base 2/4] WORKDIR /code 0.0s
=> CACHED [base 3/4] COPY package.json package.json 0.0s
=> CACHED [base 4/4] COPY package-lock.json package-lock.json 0.0s
=> CACHED [test 1/3] RUN npm ci 0.0s
=> [test 2/3] COPY . . 0.4s
=> ERROR [test 3/3] RUN npm run test 0.9s
------
> [test 3/3] RUN npm run test:
#11 0.713
#11 0.713 > nodewithtests@1.0.0 test
#11 0.713 > mocha ./**/*.js
#11 0.713
#11 0.876
#11 0.876
#11 0.877 Array
#11 0.877 #indexOf()
#11 0.879 1) should return -1 when the value is missing
#11 0.880
#11 0.880
#11 0.880 0 passing (5ms)
#11 0.880 1 failing
#11 0.880
#11 0.882 1) Array
#11 0.882 #indexOf()
#11 0.882 should return -1 when the value is missing:
#11 0.882
#11 0.882 AssertionError [ERR_ASSERTION]: 2 == -1
#11 0.882 + expected - actual
#11 0.882
#11 0.882 -2
#11 0.882 +-1
#11 0.882
#11 0.882 at Context.<anonymous> (tests/test.js:5:14)
#11 0.882 at processImmediate (node:internal/timers:466:21)
#11 0.882
#11 0.882
#11 0.882
------
executor failed running [/bin/sh -c npm run test]: exit code: 1
Don’t forget to change it back
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ !v
vi tests/test.js
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ cat tests/test.js
var assert = require('assert');
describe('Array', function() {
describe('#indexOf()', function() {
it('should return -1 when the value is missing', function() {
assert.equal([1, 2, 3].indexOf(4), -1);
});
});
});
Now that we covered building the tests in Docker, let’s update the Github workflow
GH to Docker build
Here we will switch over to using GH to build Docker.
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ vi .github/workflows/main.yaml
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ cat .github/workflows/main.yaml
name: Testing
on:
push:
branches: main
pull_request:
branches: main
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Check Out Repo
uses: actions/checkout@v3
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v1
- name: Build Only
id: docker_build
uses: docker/build-push-action@v2
with:
context: ./
file: ./Dockerfile
push: false
target: test
tags: idjohnson/dockerwithtests:latest
- name: Image digest
run: echo $
You’ll notice nowhere do I install and run NodeJS now
Let’s add a push the changes
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git add Dockerfile
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git add .github/workflows/main.yaml
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git commit -m "Docker with Tests, update GH WF"
[main 469080f] Docker with Tests, update GH WF
2 files changed, 31 insertions(+), 11 deletions(-)
create mode 100644 Dockerfile
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git push
Enumerating objects: 10, done.
Counting objects: 100% (10/10), done.
Delta compression using up to 16 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (6/6), 894 bytes | 894.00 KiB/s, done.
Total 6 (delta 1), reused 0 (delta 0)
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
To https://github.com/idjohnson/dockerWithTests2.git
b573b35..469080f main -> main
And we can see that builds in Github without issue
Now if we try and push to Dockerhub
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ vi .github/workflows/main.yaml
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git diff
diff --git a/.github/workflows/main.yaml b/.github/workflows/main.yaml
index f2a07ff..b58fd64 100644
--- a/.github/workflows/main.yaml
+++ b/.github/workflows/main.yaml
@@ -24,7 +24,7 @@ jobs:
with:
context: ./
file: ./Dockerfile
- push: false
+ push: true
target: test
tags: idjohnson/dockerwithtests:latest
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git add .github/workflows/main.yaml
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git commit -m "push"
[main 85c4443] push
1 file changed, 1 insertion(+), 1 deletion(-)
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ git push
Enumerating objects: 9, done.
Counting objects: 100% (9/9), done.
Delta compression using up to 16 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (5/5), 391 bytes | 391.00 KiB/s, done.
Total 5 (delta 2), reused 0 (delta 0)
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To https://github.com/idjohnson/dockerWithTests2.git
469080f..85c4443 main -> main
You can expect an error (as we are not authed to Dockerhub)
> exporting to image:
------
error: failed to solve: server message: insufficient_scope: authorization failed
Error: buildx failed with: error: failed to solve: server message: insufficient_scope: authorization failed
We can create a proper Auth token in Dockerhub
We just need write and read
Now just copy the token
We will use it for the value of DOCKERHUB_TOKEN in Github Repo’s settings in the Security/Secrets area:
Actually, I realized as a public repo, I may not wish to expose to any branches.
Thus, I created a “dev” environment and set the values there:
Then we add a block for the Dockerhub login
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ !v
vi .github/workflows/main.yaml
builder@DESKTOP-QADGF36:~/Workspaces/dockerWithTests2$ cat .github/workflows/main.yaml
name: Testing
on:
push:
branches: main
pull_request:
branches: main
jobs:
build:
runs-on: ubuntu-latest
environment: dev
steps:
- name: Check Out Repo
uses: actions/checkout@v3
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v1
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: '$'
password: '$'
- name: Build Only
id: docker_build
uses: docker/build-push-action@v2
with:
context: ./
file: ./Dockerfile
push: true
target: test
tags: idjohnson/dockerwithtests:latest
- name: Image digest
run: echo $
We can see it was successful
And our container is now out there for folks to use: https://hub.docker.com/repository/docker/idjohnson/dockerwithtests
Wrapping
Since we created a Github Environment for the purpose of protection, let’s add a PR policy to ensure it’s not used for evil
Summary
We walked through creating a simple NodeJS app using the Ronin Framework. Much of this, albeit dated, is covered in a Docker hosted walkthrough.
However, I felt it was worth working through and updating, using later NodeJS versions, adding Github workflows and showing some positive and negative runs to get an idea what we can do. We can see how we can extend Ronin for databases in other repos as well.
This pattern works for more than just NodeJS. For instance, we can see from this SO thread using PyTest
FROM python:3.7.6 AS build
WORKDIR /app
COPY requirements.txt .
RUN pip3 install --compile -r requirements.txt && rm -rf /root/.cache
COPY src /app
# TODO precompile
# Build stage test - run tests
FROM build AS test
RUN pip3 install pytest pytest-cov && rm -rf /root/.cache
RUN pytest --doctest-modules \
--junitxml=xunit-reports/xunit-result-all.xml \
--cov \
--cov-report=xml:coverage-reports/coverage.xml \
--cov-report=html:coverage-reports/
# Build stage 3 - Complete the build setting the executable
FROM build AS final
CMD [ "python", "./service.py" ]
Or as in this blog usign GoLang
FROM golang:1.12-alpine
RUN set -ex; \
apk update; \
apk add --no-cache git
WORKDIR /go/src/github.com/george-e-shaw-iv/integration-tests-example/
CMD CGO_ENABLED=0 go test ./...
We will likely build on this pattern as we dig into GitOps; how we can orchestrate a workflow without needing any CI application layer to handle our compilation and testing.
