Published: Jan 8, 2021 by Isaac Johnson
One of the nice features of Azure DevOps Repos is the ability to control Pull Requests with PR Policies. Policies such as minimum number of reviewers, PR build checks, comment response and Work Item linking are great, however those looking to implement custom checks (like Description keywords or tags in Titles) are often in need of further validation options.
We can use a PR server and Pull Request Status validation policies to enforce compliance with custom rules we manage and control outside of Azure DevOps itself.
Setup
Setup a new Node Package
$ nvm use 10.22.1
Now using node v10.22.1 (npm v6.14.6)
Next we npm init to make this a node app.
$ npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sensible defaults.
See `npm help init` for definitive documentation on these fields
and exactly what they do.
Use `npm install <pkg>` afterwards to install a package and
save it as a dependency in the package.json file.
Press ^C at any time to quit.
package name: (azdoprstatus)
version: (1.0.0)
description: PRStatus
entry point: (index.js) app.js
test command:
git repository:
keywords: AzDO
author: Isaac Johnson
license: (ISC) MIT
About to write to /home/builder/Workspaces/AzDOPRStatus/package.json:
{
"name": "azdoprstatus",
"version": "1.0.0",
"description": "PRStatus",
"main": "app.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [
"AzDO"
],
"author": "Isaac Johnson",
"license": "MIT"
}
Is this OK? (yes) yes
Install Express
$ npm install --save express
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN azdoprstatus@1.0.0 No repository field.
+ express@4.17.1
added 50 packages from 37 contributors and audited 50 packages in 1.774s
found 0 vulnerabilities
╭─────────────────────────────────────────────────────────────────╮
│ │
│ New patch version of npm available! 6.14.6 → 6.14.10 │
│ Changelog: https://github.com/npm/cli/releases/tag/v6.14.10 │
│ Run npm install -g npm to update! │
│ │
╰─────────────────────────────────────────────────────────────────╯
Create an app.js file with contents
const express = require('express')
const app = express()
app.get('/', function (req, res) {
res.send('Hello World!')
})
app.listen(3000, function () {
console.log('Example app listening on port 3000!')
})
e.g.
$ cat > app.js <<EOF
> const express = require('express')
> const app = express()
>
> app.get('/', function (req, res) {
> res.send('Hello World!')
> })
app.lis>
> app.listen(3000, function () {
ole.log(> console.log('Example app listening on port 3000!')
> })
> EOF
Test it
Add a line to note POST received
$ cat >> app.js <<EOF
>
> app.post('/', function (req, res) {
> res.send('Received the POST')
> })
> EOF
Test it
###
Exposing a public ingress
Download and install ngrok
Unzip then copy locally for WSL
builder@DESKTOP-JBA79RT:~$ cp /mnt/c/Users/isaac/Downloads/ngrok-stable-linux-amd64/ngrok /usr/local/bin/
cp: cannot create regular file '/usr/local/bin/ngrok': Permission denied
builder@DESKTOP-JBA79RT:~$ sudo cp /mnt/c/Users/isaac/Downloads/ngrok-stable-linux-amd64/ngrok /usr/local/bin/
[sudo] password for builder:
builder@DESKTOP-JBA79RT:~$ sudo chmod u+x /usr/local/bin/ngrok
Verify it works
Check our version
$ ngrok --version
ngrok version 2.3.35
Get the token from ngrok:
save the token
$ ngrok authtoken *************
Authtoken saved to configuration file: /home/builder/.ngrok2/ngrok.yml
Then launch ngrok http 3000
Now we can curl -X POST <ngrok URL> to test
Setting up Service Hooks in AzDO
First, I had the strangest thing happen in that the new project I had created of which I was owner in an organization of which i was owner would not let me set a Service Hook. It claimed I did not have sufficient permission.
If this happens to you, ensure you have “Edit Project-level information” set. Mine was set to “Allow (inherited)”, but clearly it wasn’t allowing me. I changed to “Allow” (explicit) and that fixed things.
Create a subscription in a Project
Choose webhook
We will want to use “Pull request creaeted”
Enter in your URL you got from ngrok then click test
Testing should show it triggered both in Azure DevOps and in the ngrok window
And in that Test Notification window, we can click on the “Response” tab to see it got 200 status back
Click Finish
And we should see it’s now live:
Allowing PR Writebacks
Next we need to actually change this PR server to send status back to Azure DevOps
After you’ve turned off ngrok and npm server, go ahead and add azure-devops-node-api and body-parser packages
$ npm install --save azure-devops-node-api body-parser
npm WARN azdoprstatus@1.0.0 No repository field.
+ body-parser@1.19.0
+ azure-devops-node-api@10.2.0
added 5 packages from 9 contributors, updated 1 package and audited 56 packages in 0.976s
1 package is looking for funding
run `npm fund` for details
found 0 vulnerabilities
We’ll do the rest of the edits in VS Code.
builder@DESKTOP-JBA79RT:~/Workspaces/AzDOPRStatus$ code .
the app.js file:
const vsts = require("azure-devops-node-api")
const bodyParser = require('body-parser')
app.use(bodyParser.json())
const collectionURL = process.env.COLLECTIONURL;
const token = process.env.TOKEN;
var authHandler = vsts.getPersonalAccessTokenHandler(token);
var connection = new vsts.WebApi(collectionURL, authHandler);
var vstsGit = null;
connection.getGitApi().then((api) => { vstsGit = api; })
app.get('/', function (req, res) {
res.send('Hello World!')
})
app.listen(3000, function () {
console.log('Example app listening on port 3000!')
})
app.post("/", function (req, res) {
// Get the details about the PR from the service hook payload
var repoId = req.body.resource.repository.id
var pullRequestId = req.body.resource.pullRequestId
var title = req.body.resource.title
console.log("repoID: " + repoId )
console.log("pullRequestId: " + pullRequestId )
console.log("title: " + title )
// Build the status object that we want to post.
// Assume that the PR is ready for review...
var prStatus = {
"state": "succeeded",
"description": "wip-checker",
"targetUrl": "https://visualstudio.microsoft.com",
"context": {
"name": "wip-checker",
"genre": "continuous-integration"
}
}
// Check the title to see if there is "WIP" in the title.
if (title.includes("WIP")) {
console.log("We got a winner!")
// If so, change the status to pending and change the description.
prStatus.state = "pending"
prStatus.description = "wip-checker"
}
// Post the status to the PR
vstsGit.createPullRequestStatus(prStatus, repoId, pullRequestId).then( result => {
console.log(result)
});
res.send("Received the POST")
})
To test, we need to set our CollectionURL and Token (PAT). If you need a PAT for this project, you can find/create them in the “Personal Access Tokens” of your account
We can now launch the app
And if needbe, restart ngrok
Since i did restart ngrok, I’ll need to go to the service hooks to update the URL
Change to the URL.
Note, if you test, you may see an error:
$ node app.js
Example app listening on port 3000!
repoID: 4bc14d40-c903-45e2-872e-0462c7748079
pullRequestId: 1
title: my first pull request
TypeError: vstsGit.createPullRequestStatus is not a function
at /home/builder/Workspaces/AzDOPRStatus/app.js:57:13
at Layer.handle [as handle_request] (/home/builder/Workspaces/AzDOPRStatus/node_modules/express/lib/router/layer.js:95:5)
Because the example status isn’t a real PR nor a real Repo ID
Testing
First, lets make an edit on the README.md
Then click commit and save it to a new branch and PR
The PR
Quick note: if you are testing changes and it fails a lot, the service connection can become disabled. Go back to service connections to re-enable if you suspect it’s not being triggered due to too many failures:
We can see it set to “Running” (pending) if the title has [WIP]
And changing to remove “[WIP]” then updates that check (provided you added a ‘PR updated’ webhook event, see next section).
A note…
I had a lot of troubles getting the demo code to work.. It seems the latest azure CLI node package either doesnt work for Node 10.x (maybe there are some node 12 assumptions) or its got a bug on PR update:
(node:9399) UnhandledPromiseRejectionWarning: Error: TF400813: The user '' is not authorized to access this resource.
at RestClient.<anonymous> (/home/builder/Workspaces/AzDOPRStatus/node_modules/typed-rest-client/RestClient.js:202:31)
at Generator.next (<anonymous>)
at fulfilled (/home/builder/Workspaces/AzDOPRStatus/node_modules/typed-rest-client/RestClient.js:6:58)
at process._tickCallback (internal/process/next_tick.js:68:7)
(node:9399) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:9399) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
In working it out, i found the older version (9.0.1) works well with node 10.22.1 which i use:
"azure-devops-node-api": "^9.0.1",
"body-parser": "^1.19.0",
"express": "^4.17.1"
And from package-lock.json:
"azure-devops-node-api": {
"version": "9.0.1",
"resolved": "https://registry.npmjs.org/azure-devops-node-api/-/azure-devops-node-api-9.0.1.tgz",
"integrity": "sha512-0veE4EWHObJxzwgHlydG65BjNMuLPkR1nzcQ2K51PIho1/F4llpKt3pelC30Vbex5zA9iVgQ9YZGlkkvOBSksw==",
"requires": {
"tunnel": "0.0.4",
"typed-rest-client": "1.2.0",
"underscore": "1.8.3"
}
},
Another example.
Take a look at this “Checklist checker”: https://github.com/jagdish7908/Check
Download and install the npm packages
$ git clone https://github.com/jagdish7908/Check.git
Cloning into 'Check'...
remote: Enumerating objects: 723, done.
remote: Total 723 (delta 0), reused 0 (delta 0), pack-reused 723
Receiving objects: 100% (723/723), 1.14 MiB | 5.10 MiB/s, done.
Resolving deltas: 100% (169/169), done.
builder@DESKTOP-JBA79RT:~/Workspaces$ cd Check/
builder@DESKTOP-JBA79RT:~/Workspaces/Check$ nvm use 10.22.1
Now using node v10.22.1 (npm v6.14.6)
$ npm install
added 54 packages from 45 contributors and audited 54 packages in 0.859s
found 0 vulnerabilities
Set our env vars and run:
builder@DESKTOP-JBA79RT:~/Workspaces/Check$ export COLLECTIONURL="https://dev.azure.com/princessking"
builder@DESKTOP-JBA79RT:~/Workspaces/Check$ export TOKEN= ******************************
builder@DESKTOP-JBA79RT:~/Workspaces/Check$ export PORT=3000
builder@DESKTOP-JBA79RT:~/Workspaces/Check$ node app.js
Checklist checker listening on port 3000
Now when we create a PR, we can see the lack of a checklist fails the PR:
$ node app.js
Checklist checker listening on port 3000
{ id: 1,
state: 3,
description: 'Checklist',
context:
{ name: 'checklist-checker', genre: 'continuous-integration' },
creationDate: 2021-01-07T17:03:48.772Z,
updatedDate: 2021-01-07T17:03:48.772Z,
createdBy:
{ displayName: 'Isaac Johnson',
url:
…..
However, a small issue.. When we edit…
It still shows failed. That’s because we need to update our Service Hook for _more_ than just the created event
Add a webhook for updated as well (to same URL)
Now when we edit, we see it re-evaluate to good:
Note: Updated events are not inclusive of created, so you would need both rules for PRs.
Enforcement
You’ll notice that the PR update happens asynchronously after the PR is created. This means one could “sneak” in a PR before the wip-checker indicated it was satisfactory. To avoid this, we can use a PR policy to enforce compliance.
We can set it for main any repo in our project (Project policies):
This requires wip-checker to indicate “succeeded” as a blocking gate on PRs for main.
Now when I create a PR in that project and indicate WIP:
I will see immediately that “wip-checker” was not run and it cannot merge yet:
In a moment, the webhook is invoked and we see the “pending” status (animated arrows):
If I edit the title and save:
In a moment the webhook runs again and the check is satisfied
Summary
In this demo we dug into a couple examples of nodejs based PR service connection servers. Using the vsts API we served up an http service and proved we could decorate PRs with it. We then extended this to leverage PR policies to force compliance to external PR validations.
These patterns make it easy to extend the Pull Request capabilities to add custom checks to ensure compliance. Next steps would be to containerize the service and expose it via an HTTP ingress. We would also want to add some form of token to ensure the webhook access is restricted.
