Whitesource Bolt for AzDO and Github
On recommendations from a colleague, I decided to checkout Whitesource Bolt which was recently made free for developers and demo’ed at Microsoft Build 2020. It integrates natively with Azure DevOps and Github.
They have a commercial offering that starts at $1260/user for “Whitesource for Developers” and $4200/user for “Whitesource Core”. However, they do have this free offering. I have to admit the pricing throws me (seems like American Pharma pricing with free samples). I have reached out for a demo/commercial demo license and if that plays out, I’ll followup this review with those findings.
Adding to Azure DevOps
Go to the Marketplace to add the plugin: https://marketplace.visualstudio.com/items?itemName=whitesource.ws-bolt
And then click “Get it free” to install it into your organization:
We should now see it in Settings/Extensions:
Using with NodeJS
Let’s start by doing this in a fork:
C:\Users\isaac\Workspaces\ghost-blog>git checkout -b feature/enable-whitesource-bolt
M node_modules/.bin/atob
M node_modules/.bin/bunyan
M node_modules/.bin/css-beautify
…
Next, lets add a step to our pipeline to scan:
- task: whitesource.ws-bolt.bolt.wss.WhiteSource Bolt@20
displayName: 'WhiteSource Bolt'
inputs:
cwd: '$(Pipeline.Workspace)'Next add and push:
C:\Users\isaac\Workspaces\ghost-blog>git add azure-pipelines.yml
C:\Users\isaac\Workspaces\ghost-blog>git commit -m "Add Whitesource Bolt"
[feature/enable-whitesource-bolt d7cee75c] Add Whitesource Bolt
1 file changed, 5 insertions(+)
C:\Users\isaac\Workspaces\ghost-blog>git push --set-upstream origin feature/enable-whitesource-bolt
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 389 bytes | 194.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Analyzing objects... (3/3) (781 ms)
remote: Storing packfile... done (191 ms)
remote: Storing index... done (68 ms)
remote: We noticed you're using an older version of Git. For the best experience, upgrade to a newer version.
To https://princessking.visualstudio.com/ghost-blog/_git/ghost-blog
* [new branch] feature/enable-whitesource-bolt -> feature/enable-whitesource-bolt
Branch 'feature/enable-whitesource-bolt' set up to track remote branch 'feature/enable-whitesource-bolt' from 'origin'.
Next, we can either create a PR which will trigger a build Or we can manually invoke a build on this branch (and our release steps should be skipped due to the branch restrictions).
Manual Build
Choose Run Pipeline:
Then change the branch and choose run:
If we want, we can click “Stages to run” and manually remove the release stages (though I trust the conditions to not run so i’ll leave them be):
If we just run it now, we’ll get an error:
2020-05-25T14:45:02.7683324Z ##[section]Starting: WhiteSource Bolt
2020-05-25T14:45:02.7690177Z ==============================================================================
2020-05-25T14:45:02.7690474Z Task : WhiteSource Bolt
2020-05-25T14:45:02.7690790Z Description : Detect & fix security vulnerabilities, problematic open source licenses.
2020-05-25T14:45:02.7691093Z Version : 20.5.1
2020-05-25T14:45:02.7691290Z Author : WhiteSource
2020-05-25T14:45:02.7691539Z Help : http://www.whitesourcesoftware.com
2020-05-25T14:45:02.7691832Z ==============================================================================
2020-05-25T14:45:03.0790674Z (node:4269) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-05-25T14:45:03.0791264Z (node:4269) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-05-25T14:45:03.0791753Z (node:4269) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-05-25T14:45:03.0792255Z (node:4269) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-05-25T14:45:03.0792844Z (node:4269) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-05-25T14:45:03.0793344Z (node:4269) Warning: Use Cipheriv for counter mode of aes-256-ctr
2020-05-25T14:45:03.5787750Z WhiteSource Bolt hasn't been activated in collection 97b8c412-5dbf-4532-ba2d-4c54232b59b3. To activate please place a valid code in the WhiteSource Bolt hub under "Build and Release" tab.
2020-05-25T14:45:03.5788741Z Exiting Bolt build task.
2020-05-25T14:45:03.5837257Z ##[section]Finishing: WhiteSource Bolt
It’s not actually in "Build and Release" but rather Pipelines where we need to setup the plugin:
When done
Now we can see results after a run:
Let’s take a look at some:
I then tried to mitigate these. E.g. I upgraded lodash to ^4.17.12:
I even removed any and all “node_modules” that had errantly been checked in (which did speed up the build, but didn’t fix the error).
The problem is that without line numbers, it’s rather hard to figure out where to change.
License Checks
Another really good feature of Whitesource Bolt is to check the Licenses:
If my software was commercial, I would need to honor the GPL and opensource it.
That’s where the Inventor section can help identify which software is using GPL:
This would mean i would need to hunt down that js in my app and since only the minified version is GPL’ed, I could minify it myself.
Go Lang
I tried to point it to a golang build i have. I tried at the parent dir and the folder containing my go code, but neither would show results:
And lest you think my golang has no OSS components:
You can see several imports from github.com I would have expected flagged.
Dotnet Core
Let’s build a simple dotnet core app.
It’s easy to instantiate a new project
$ dotnet new console -o myApp
Welcome to .NET Core 3.1!
---------------------
SDK Version: 3.1.300
Telemetry
---------
The .NET Core tools collect usage data in order to help us improve your experience. The data is anonymous. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.
Read more about .NET Core CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry
----------------
Explore documentation: https://aka.ms/dotnet-docs
Report issues and find source on GitHub: https://github.com/dotnet/core
Find out what's new: https://aka.ms/dotnet-whats-new
Learn about the installed HTTPS developer cert: https://aka.ms/aspnet-core-https
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli-docs
Write your first app: https://aka.ms/first-net-core-app
--------------------------------------------------------------------------------------
Getting ready...
The template "Console Application" was created successfully.
Processing post-creation actions...
Running 'dotnet restore' on myApp/myApp.csproj...
Determining projects to restore...
Restored /Users/johnsi10/Workspaces/Dotnet-Core/myApp/myApp.csproj (in 113 ms).
Restore succeeded.
And then add a 3rdParty library:
$ dotnet add package YamlDotNet --version 8.1.1
Determining projects to restore...
Writing /var/folders/dp/wgg0qtcs2lv7j0vwx4fnrgq80000gp/T/tmp4UgGOT.tmp
info : Adding PackageReference for package 'YamlDotNet' into project '/Users/johnsi10/Workspaces/Dotnet-Core/myApp/myApp.csproj'.
info : Restoring packages for /Users/johnsi10/Workspaces/Dotnet-Core/myApp/myApp.csproj...
info : GET https://api.nuget.org/v3-flatcontainer/yamldotnet/index.json
info : OK https://api.nuget.org/v3-flatcontainer/yamldotnet/index.json 36ms
info : GET https://api.nuget.org/v3-flatcontainer/yamldotnet/8.1.1/yamldotnet.8.1.1.nupkg
info : OK https://api.nuget.org/v3-flatcontainer/yamldotnet/8.1.1/yamldotnet.8.1.1.nupkg 24ms
info : Installing YamlDotNet 8.1.1.
info : Package 'YamlDotNet' is compatible with all the specified frameworks in project '/Users/johnsi10/Workspaces/Dotnet-Core/myApp/myApp.csproj'.
info : PackageReference for package 'YamlDotNet' version '8.1.1' added to file '/Users/johnsi10/Workspaces/Dotnet-Core/myApp/myApp.csproj'.
info : Committing restore...
info : Writing assets file to disk. Path: /Users/johnsi10/Workspaces/Dotnet-Core/myApp/obj/project.assets.json
log : Restored /Users/johnsi10/Workspaces/Dotnet-Core/myApp/myApp.csproj (in 835 ms).
To use this library we can update Program.cs:
$ cat Program.cs
using System;
using System.IO;
using YamlDotNet.Serialization;
namespace myApp
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Hello World!");
Console.WriteLine("The current time is " + DateTime.Now);
var r = new StringReader(@"
scalar: my scalar
sequence:
- one
- two
- three
");
var deserializer = new DeserializerBuilder().Build();
var yamlObject = deserializer.Deserialize(r);
var serializer = new SerializerBuilder()
.JsonCompatible()
.Build();
var json = serializer.Serialize(yamlObject);
Console.WriteLine(json);
}
}
}
$ dotnet run
Hello World!
The current time is 5/26/2020 9:19:38 AM
{"scalar": "my scalar", "sequence": ["one", "two", "three"]}
We can now fire a build with Whitesource Bolt:
$ cat azure-pipelines.yml
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- script: echo Hello, world!
displayName: 'Run a one-line script'
- script: |
echo Add other tasks to build, test, and deploy your project.
echo See https://aka.ms/yaml
displayName: 'Run a multi-line script'
- task: DotNetCoreCLI@2
inputs:
command: 'build'
projects: '**/*.csproj'
- task: WhiteSource Bolt@20
inputs:
cwd: 'myApp'
Results
I did need to run WS Bolt after my build step, but it found the library and indicated no issues.
Moving the scan before compile proves that:
Summary
The addition of WS Bolt scanning seemed to add between 47s and 1'40" to the build. Even if all one watches for are GPL licenses, I think that's a pretty good value.
The question is whether the "free license" is for commercial use or not (or if it's free for open-source/'self employed' developers).
The commercial pricing:
Seems rather high for an annually licensed product. But again, I'll reserve judgement until I can see a demo (However, at those prices we are in JFrog Enterprise and PRISMA levels and with PRISMA i get Twistlock and JFrog includes Xray as part of a full artifact management suite).