Azure DevOps (VSTS) security and policies (part 4)

In our last post we covered the Pull Request process and problems with leaving merged feature branches.  But what about validating the code in a pull request before merging? What does AzDO offer by way of PR pre-merge validation?  The answer is Pull Request builds for which we can leverage in PR policies as well.

Pull Request Builds

Once we are ready to build code, we create Build and Release Pipelines in Azure DevOps Pipelines.  We can trigger Continuous Integration through a CI trigger in our Build Jobs and we can use Pipelines to validate code prior to merging as well.

Example:

Let’s create a quick React App:

1. npm init react-app ./test-react-app
2. cd test-react-app
3. npm install --save jsonlint
4. Create a quick testing.json:

{
  "test" : "value",
  "another" : "value2"
}

5. Change line 7 , the test script, in the package.json to test our file:

"test": "jsonlint ./testing.json"

6. Testing should now work

C:\Workspaces\idj-ta-example>npm test

> idj-ta-example@1.0.0 test C:\Workspaces\idj-ta-example
> jsonlint ./testing.json

{
  "test": "value",
  "another": "value2"
}

Conversely, if we mangle the json, we should expect broken tests:

C:\Users\isaac\Documents\Workspaces\idj-ta-example>npm test

> idj-ta-example@1.0.0 test C:\Users\isaac\Documents\Workspaces\idj-ta-example
> jsonlint ./testing.json

Error: Parse error on line 3:
} "another" : "value2
----------------------^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got 'undefined'
    at Object.parseError (C:\Users\isaac\Documents\Workspaces\idj-ta-example\node_modules\jsonlint\lib\jsonlint.js:55:11)
    at Object.parse (C:\Users\isaac\Documents\Workspaces\idj-ta-example\node_modules\jsonlint\lib\jsonlint.js:132:22)
    at parse (C:\Users\isaac\Documents\Workspaces\idj-ta-example\node_modules\jsonlint\lib\cli.js:82:14)
    at main (C:\Users\isaac\Documents\Workspaces\idj-ta-example\node_modules\jsonlint\lib\cli.js:135:14)
    at Object.<anonymous> (C:\Users\isaac\Documents\Workspaces\idj-ta-example\node_modules\jsonlint\lib\cli.js:179:1)
    at Module._compile (internal/modules/cjs/loader.js:688:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:699:10)
    at Module.load (internal/modules/cjs/loader.js:598:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:537:12)
    at Function.Module._load (internal/modules/cjs/loader.js:529:3)
npm ERR! Test failed.  See above for more details.

Now we can add and push it to the branch, right?

This works, however, we now have a branch that is out of date (it’s missing the merge back) and restores to a used branch name!

Of course, i immediately deleted the branch (trash can icon in red above).  So we can push to a new branch, i’ll want to go back to develop, update, then create a new branch and merge these changes into it:

C:\Users\isaac\Documents\Workspaces\idj-ta-example>git checkout develop
Switched to branch 'develop'
Your branch is ahead of 'origin/develop' by 1 commit.
  (use "git push" to publish your local commits)

C:\Users\isaac\Documents\Workspaces\idj-ta-example>git pull
remote: Azure Repos
remote: Found 1 objects to send. (0 ms)
Unpacking objects: 100% (1/1), done.
From https://ijohnson-thinkahead.visualstudio.com/idj-ta-example/_git/idj-ta-example
   00c4814..277dcd0  develop    -> origin/develop
Updating b16610d..277dcd0
Fast-forward

C:\Users\isaac\Documents\Workspaces\idj-ta-example>git checkout -b feature/1234-new-react-app
Switched to a new branch 'feature/1234-new-react-app'

C:\Users\isaac\Documents\Workspaces\idj-ta-example>git merge feature/1234-a-quick-change
Merge made by the 'recursive' strategy.
 package-lock.json | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 package.json      | 18 ++++++++++++++++
 testing.json      |  4 ++++
 3 files changed, 83 insertions(+)
 create mode 100644 package-lock.json
 create mode 100644 package.json
 create mode 100644 testing.json

Let’s now push that up so we have some code to build a pipeline around;

git push --set-upstream origin feature/1234-new-react-app

Let’s create a quick pipeline that builds the code

1. Pipelines/Builds click +New
2. Create a Node based build on the repo and develop branch (I chose "Node with Grunt", but i don’t plan to create a gruntfile right now)
3. Add a step to fire “npm test”
4. Fire a test build against our branch (use Queue).

We can see with a broken testfile.json, it properly generates an error:

If we update the file to be proper json again, the build should pass tests:

Validating Pull Requests with PR builds

Now that we have a way to test the code in a build, we can set it up to validate each commit to any active PR (called an "Update" in a PR).

1. Go to Repositories, our develop branch (we set the policy on the ‘destination’ branch’) and choose policies
2. In Build Validation, choose “+ Add build policy”
3. In the “Add build policy” window, add a required build.  

Once saved, we see the PR build listed:

This means we can validate our PRs now with a build.

Testing:

Let’s use the standard user and go to the repository - we are prompted to create a PR:

Once created, we see a PR build is automatically queued:

This should pass:

However, what happens if we break the file again?  It should block the PR, even if others approve:

We see that it was approved by the Reviewer (in the left hand history) but then when I pushed another change (listed as “update 2”), it reset the vote and triggered a PR build.

That failed as i added a mangled json:

When we look at the build history, PR builds are immediately called out.  The “branch” column lists PR number instead of the root branch.  This lets us know that it was built for the purpose of validating a PR versus a CI trigger on branches.

Summary:

The point here is we can validate all PR code through a build process. This allows us to  exercise the code in whatever ways are meaningful. In this case, I added a basic linter to the npm test which would easily catch json issues and automatically prevent them from being merged into the mainline “develop” branch.  You can expand on this to do proper unit testing with Mocha / Chai.