Azure DevOps (VSTS) security and policies (part 4)

Published: Mar 2, 2019 by Isaac Johnson

In our last post we covered the Pull Request process and problems with leaving merged feature branches.  But what about validating the code in a pull request before merging? What does AzDO offer by way of PR pre-merge validation?  The answer is Pull Request builds for which we can leverage in PR policies as well.

Pull Request Builds

Once we are ready to build code, we create Build and Release Pipelines in Azure DevOps Pipelines.  We can trigger Continuous Integration through a CI trigger in our Build Jobs and we can use Pipelines to validate code prior to merging as well.


Let’s create a quick React App:

  1. npm init react-app ./test-react-app
  2. cd test-react-app
  3. npm install –save jsonlint
  4. Create a quick testing.json:

    { “test” : “value”, “another” : “value2” }

  5. Change line 7 , the test script, in the package.json to test our file:

    “test”: “jsonlint ./testing.json”

  6. Testing should now work

    C:\Workspaces\idj-ta-example>npm test

    idj-ta-example@1.0.0 test C:\Workspaces\idj-ta-example jsonlint ./testing.json

    { “test”: “value”, “another”: “value2” }

Conversely, if we mangle the json, we should expect broken tests:

C:\Users\isaac\Documents\Workspaces\idj-ta-example>npm test

> idj-ta-example@1.0.0 test C:\Users\isaac\Documents\Workspaces\idj-ta-example
> jsonlint ./testing.json

Error: Parse error on line 3:
} "another" : "value2
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got 'undefined'
    at Object.parseError (C:\Users\isaac\Documents\Workspaces\idj-ta-example\node_modules\jsonlint\lib\jsonlint.js:55:11)
    at Object.parse (C:\Users\isaac\Documents\Workspaces\idj-ta-example\node_modules\jsonlint\lib\jsonlint.js:132:22)
    at parse (C:\Users\isaac\Documents\Workspaces\idj-ta-example\node_modules\jsonlint\lib\cli.js:82:14)
    at main (C:\Users\isaac\Documents\Workspaces\idj-ta-example\node_modules\jsonlint\lib\cli.js:135:14)
    at Object.<anonymous> (C:\Users\isaac\Documents\Workspaces\idj-ta-example\node_modules\jsonlint\lib\cli.js:179:1)
    at Module._compile (internal/modules/cjs/loader.js:688:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:699:10)
    at Module.load (internal/modules/cjs/loader.js:598:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:537:12)
    at Function.Module._load (internal/modules/cjs/loader.js:529:3)
npm ERR! Test failed. See above for more details.

Now we can add and push it to the branch, right?

This works, however, we now have a branch that is out of date (it’s missing the merge back) and restores to a used branch name!

Branches view in our Repo

Of course, i immediately deleted the branch (trash can icon in red above).  So we can push to a new branch, i’ll want to go back to develop, update, then create a new branch and merge these changes into it:

C:\Users\isaac\Documents\Workspaces\idj-ta-example>git checkout develop
Switched to branch 'develop'
Your branch is ahead of 'origin/develop' by 1 commit.
  (use "git push" to publish your local commits)

C:\Users\isaac\Documents\Workspaces\idj-ta-example>git pull
remote: Azure Repos
remote: Found 1 objects to send. (0 ms)
Unpacking objects: 100% (1/1), done.
   00c4814..277dcd0 develop -> origin/develop
Updating b16610d..277dcd0

C:\Users\isaac\Documents\Workspaces\idj-ta-example>git checkout -b feature/1234-new-react-app
Switched to a new branch 'feature/1234-new-react-app'

C:\Users\isaac\Documents\Workspaces\idj-ta-example>git merge feature/1234-a-quick-change
Merge made by the 'recursive' strategy.
 package-lock.json | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 package.json | 18 ++++++++++++++++
 testing.json | 4 ++++
 3 files changed, 83 insertions(+)
 create mode 100644 package-lock.json
 create mode 100644 package.json
 create mode 100644 testing.json

Let’s now push that up so we have some code to build a pipeline around;

git push --set-upstream origin feature/1234-new-react-app

Let’s create a quick pipeline that builds the code

  1. Pipelines/Builds click +New
  2. Create a Node based build on the repo and develop branch (I chose “Node with Grunt”, but i don’t plan to create a gruntfile right now)
  3. Add a step to fire “npm test”
  4. Fire a test build against our branch (use Queue).

We can see with a broken testfile.json, it properly generates an error:

A build in error due to linting
Log details on test step

If we update the file to be proper json again, the build should pass tests:

Updating the text file and pushing
Validation from a manually triggered build

Validating Pull Requests with PR builds

Now that we have a way to test the code in a build, we can set it up to validate each commit to any active PR (called an “Update” in a PR).

  1. Go to Repositories, our develop branch (we set the policy on the ‘destination’ branch’) and choose policies
  2. In Build Validation, choose “+ Add build policy”
  3. In the “Add build policy” window, add a required build.  
Note: the path trigger can be used to trim builds.. Eg. fire the .NET build for /dotnet (and sub folder changes) vs /java (for java app)

Once saved, we see the PR build listed:

PR Build Validation in the Policies Settings

This means we can validate our PRs now with a build.


Let’s use the standard user and go to the repository - we are prompted to create a PR:

PR Creation from the Repo files view

Once created, we see a PR build is automatically queued:

The play icon in the Policies section shows that a "PR Build [is] in progress"

This should pass:

A passing build

However, what happens if we break the file again?  It should block the PR, even if others approve:

Build running after an approval

We see that it was approved by the Reviewer (in the left hand history) but then when I pushed another change (listed as “update 2”), it reset the vote and triggered a PR build.

That failed as i added a mangled json:

A failed PR build

When we look at the build history, PR builds are immediately called out.  The “branch” column lists PR number instead of the root branch.  This lets us know that it was built for the purpose of validating a PR versus a CI trigger on branches.

Pipeline Build History showing PRs and Branches in the Branch Column


The point here is we can validate all PR code through a build process. This allows us to  exercise the code in whatever ways are meaningful. In this case, I added a basic linter to the npm test which would easily catch json issues and automatically prevent them from being merged into the mainline “develop” branch.  You can expand on this to do proper unit testing with Mocha / Chai.

tutorial vsts

Isaac Johnson

Isaac Johnson

Cloud Solutions Architect

Isaac is a CSA and DevOps engineer who focuses on cloud migrations and devops processes. He also is a dad to three wonderful daughters (hence the references to Princess King sprinkled throughout the blog).

Theme built by C.S. Rhymes