Published: Nov 5, 2020 by Isaac Johnson
A topic that has come up a few times lately, in regards to YAML pipelines in Azure DevOps, is how to use Environments for non-containerized workloads. Most of to whom I speak are pretty comfortable with the Kubernetes “environment” with namespaces and gates, but how do the “VirtualMachines” work in the context of deployment environments?
Let’s take a moment to clear up some confusion on this topic and do some working examples.
Creating a new Environment
Create with the Resource of Virtual Machines
The command to copy shows:
mkdir azagent;cd azagent;curl -fkSL -o vstsagent.tar.gz https://vstsagentpackage.azureedge.net/agent/2.175.2/vsts-agent-linux-x64-2.175.2.tar.gz;tar -zxvf vstsagent.tar.gz; if [-x "$(command -v systemctl)"]; then ./config.sh --environment --environmentname "DeploymentEnvironment" --acceptteeeula --agent $HOSTNAME --url https://princessking.visualstudio.com/ --work _work --projectname 'Fortran_CICD' --auth PAT --token ***************************************--runasservice; sudo ./svc.sh install; sudo ./svc.sh start; else ./config.sh --environment --environmentname "DeploymentEnvironment" --acceptteeeula --agent $HOSTNAME --url https://princessking.visualstudio.com/ --work _work --projectname 'Fortran_CICD' --auth PAT --token*************************************** ; ./run.sh; fi
It should be noted, the command DOES include the PAT - i am just masking here.
Create some VMs
We use can the Azure CLI to create one of the VMs
$ az vm create --name ijsamplelinuxvm --resource-group idjaksdemo --image UbuntuLTS --ssh-key-values @/Users/johnsi10/.ssh/id_rsa.pub
{- Finished ..
"fqdns": "",
"id": "/subscriptions/asdfasd-asdfasdfasd-asdfasdf-asdfasdf/resourceGroups/idjaksdemo/providers/Microsoft.Compute/virtualMachines/ijsamplelinuxvm",
"location": "eastus",
"macAddress": "00-0D-3A-12-C8-4A",
"powerState": "VM running",
"privateIpAddress": "10.0.0.4",
"publicIpAddress": "52.170.222.194",
"resourceGroup": "idjaksdemo",
"zones": ""
}
Verify we can login
$ ssh 52.170.222.194
The authenticity of host '52.170.222.194 (52.170.222.194)' can't be established.
ECDSA key fingerprint is SHA256:w+sSHT0n4ZxLb0MSUkQe5grN8CDxXkk63/WK7dsEx7k.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '52.170.222.194' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 5.4.0-1031-azure x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Thu Nov 5 14:37:44 UTC 2020
System load: 0.0 Processes: 109
Usage of /: 4.4% of 28.90GB Users logged in: 0
Memory usage: 5% IP address for eth0: 10.0.0.4
Swap usage: 0%
0 packages can be updated.
0 updates are security updates.
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
johnsi10@ijsamplelinuxvm:~$ sudo echo hi
hi
Install the agent
johnsi10@ijsamplelinuxvm:~$ mkdir azagent;cd azagent;curl -fkSL -o vstsagent.tar.gz https://vstsagentpackage.azureedge.net/agent/2.175.2/vsts-agent-linux-x64-2.175.2.tar.gz;tar -zxvf vstsagent.tar.gz; if [-x "$(command -v systemctl)"]; then ./config.sh --environment --environmentname "DeploymentEnvironment" --acceptteeeula --agent $HOSTNAME --url https://princessking.visualstudio.com/ --work _work --projectname 'Fortran_CICD' --auth PAT --token asdfasdfasdfasdfasdfasdfasdfasdfasdfasdf --runasservice; sudo ./svc.sh install; sudo ./svc.sh start; else ./config.sh --environment --environmentname "DeploymentEnvironment" --acceptteeeula --agent $HOSTNAME --url https://princessking.visualstudio.com/ --work _work --projectname 'Fortran_CICD' --auth PAT --token asdfasdfasdfasdfasdfasdfasdfasdfasdfasdf; ./run.sh; fi
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 130M 100 130M 0 0 183M 0 --:--:-- --:--:-- --:--:-- 183M
./
./config.sh
./bin/
./bin/Agent.Listener.deps.json
./bin/System.IO.FileSystem.Watcher.dll
….
./externals/vso-task-lib/node_modules/.bin/shjs.cmd
_________ _ _ _
/ _ \ | ___ (_) | (_)
/ /_\ \ ______ ___ ___| |_/ /_ ___ ___| |_ ___ ______
| _ |_ / | | | ' __/ _ \ |__ /| | '_ \ / _ \ | | '_ \ / _ \/ __|
| | | |/ /| |_| | | | __/ | | | | |_) |__/ | | | | | __/\__ \
\_| |_/ ___|\__ ,_|_| \ ___| \_| |_| .__ / \ ___|_|_|_| |_|\___ ||___/
| |
agent v2.175.2 |_| (commit 5c4925c)
>> End User License Agreements:
Building sources from a TFVC repository requires accepting the Team Explorer Everywhere End User License Agreement. This step is not required for building sources from Git repositories.
A copy of the Team Explorer Everywhere license agreement can be found at:
/home/johnsi10/azagent/externals/tee/license.html
>> Connect:
Connecting to server ...
>> Register Agent:
Scanning for tool capabilities.
Connecting to the server.
Enter Environment Virtual Machine resource tags? (Y/N) (press enter for N) > Y
Enter Comma separated list of tags (e.g web, db) > azurevm linux
Successfully added the agent
Testing agent connection.
2020-11-05 14:41:03Z: Settings Saved.
Creating launch agent in /etc/systemd/system/vsts.agent.princessking..ijsamplelinuxvm.service
Run as user: johnsi10
Run as uid: 1000
gid: 1000
Created symlink /etc/systemd/system/multi-user.target.wants/vsts.agent.princessking..ijsamplelinuxvm.service → /etc/systemd/system/vsts.agent.princessking..ijsamplelinuxvm.service.
/etc/systemd/system/vsts.agent.princessking..ijsamplelinuxvm.service
● vsts.agent.princessking..ijsamplelinuxvm.service - Azure Pipelines Agent (princessking..ijsamplelinuxvm)
Loaded: loaded (/etc/systemd/system/vsts.agent.princessking..ijsamplelinuxvm.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2020-11-05 14:41:06 UTC; 18ms ago
Main PID: 2064 (runsvc.sh)
Tasks: 2 (limit: 4075)
CGroup: /system.slice/vsts.agent.princessking..ijsamplelinuxvm.service
└─2064 /bin/bash /home/johnsi10/azagent/runsvc.sh
Nov 05 14:41:06 ijsamplelinuxvm systemd[1]: Started Azure Pipelines Agent (princessking..ijsamplelinuxvm).
We now see it listed
If we choose manage tags
We can see that tag we added:
Let’s also create a VM somewhere else as well:
Once logged in, create a user
root@localhost:~# adduser azdoagent
Adding user `azdoagent' ...
Adding new group `azdoagent' (1000) ...
Adding new user `azdoagent' (1000) with group `azdoagent' ...
Creating home directory `/home/azdoagent' ...
Copying files from `/etc/skel' ...
New password:
Retype new password:
passwd: password updated successfully
Changing the user information for azdoagent
Enter the new value, or press ENTER for the default
Full Name []: AzDO user
Room Number []:
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [Y/n] Y
root@localhost:~# usermod -aG sudo azdoagent
Then update sudoers sudo visudo
# Allow members of group sudo to execute any command
%sudo ALL=(ALL:ALL) NOPASSWD:ALL
Validate
root@localhost:~# su - azdoagent
azdoagent@localhost:~$ sudo echo hi
hi
Adding that agent, we’ll use other tags
….
Scanning for tool capabilities.
Connecting to the server.
Enter Environment Virtual Machine resource tags? (Y/N) (press enter for N) > Y
Enter Comma separated list of tags (e.g web, db) > linode linux
Successfully added the agent
Testing agent connection.
2020-11-05 14:56:48Z: Settings Saved.
Creating launch agent in /etc/systemd/system/vsts.agent.princessking..localhost.service
Run as user: azdoagent
Run as uid: 1000
gid: 1000
Created symlink /etc/systemd/system/multi-user.target.wants/vsts.agent.princessking..localhost.service → /etc/systemd/system/vsts.agent.princessking..localhost.service.
/etc/systemd/system/vsts.agent.princessking..localhost.service
● vsts.agent.princessking..localhost.service - Azure Pipelines Agent (princessking..localhost)
Loaded: loaded (/etc/systemd/system/vsts.agent.princessking..localhost.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2020-11-05 14:56:48 UTC; 11ms ago
Main PID: 835 (runsvc.sh)
Tasks: 2 (limit: 1149)
Memory: 476.0K
CGroup: /system.slice/vsts.agent.princessking..localhost.service
├─835 /bin/bash /home/azdoagent/azagent/runsvc.sh
└─837 /bin/bash /home/azdoagent/azagent/runsvc.sh
Nov 05 14:56:48 localhost systemd[1]: Started Azure Pipelines Agent (princessking..localhost).
The only issue is that that host just knew itself as “localhost” .. but we can use the tags to see that indeed, it is the nanode
I will note that i tried to add my Mac, but it seems the binaries are not darwin (mac os) compatible
./config.sh: line 85: ./bin/Agent.Listener: cannot execute binary file
./run.sh: line 43: /Users/johnsi10/agentdemo/azagent/bin/Agent.Listener: cannot execute binary file
Using the VM Deployment
Lets create a new repo for our pipeline
Then we can start creating our pipeline by clicking setup build
We can do a simple starter pipeline
There is the basic one:
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- script: echo Hello, world!
displayName: 'Run a one-line script'
- script: |
echo Add other tasks to build, test, and deploy your project.
echo See https://aka.ms/yaml
displayName: 'Run a multi-line script'
But we can easily add our Deployment environment:
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
jobs:
- job: build
steps:
- script: echo Hello, world!
displayName: 'Run a one-line script'
- script: |
echo Add other tasks to build, test, and deploy your project.
echo See https://aka.ms/yaml
displayName: 'Run a multi-line script'
- deployment: vmupdater
displayName: "Update VMs"
environment:
name: DeploymentEnvironment
resourceType: VirtualMachine
strategy:
rolling:
maxParallel: 2
deploy:
steps:
- bash: |
#!/bin/bash
set -x
touch /tmp/i-updated-something.txt
pwd
uname -a
displayName: 'touch a file'
on:
failure:
steps:
- script: echo FAILED
success:
steps:
- script: echo PASSED
and the output
Let’s look at some of the output on the “Deploy” stages
Linux ijsamplelinuxvm 5.4.0-1031-azure #32~18.04.1-Ubuntu SMP Tue Oct 6 10:03:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Linux localhost 4.19.0-11-amd64 #1 SMP Debian 4.19.146-1 (2020-09-17) x86_64 GNU/Linux
Lastly, if we hop on the hosts, we can see that the file was updated:
johnsi10@ijsamplelinuxvm:~$ ls -ltra /tmp
total 36
drwxrwxrwt 2 root root 4096 Nov 5 14:08 .font-unix
drwxrwxrwt 2 root root 4096 Nov 5 14:08 .XIM-unix
drwxrwxrwt 2 root root 4096 Nov 5 14:08 .X11-unix
drwxrwxrwt 2 root root 4096 Nov 5 14:08 .Test-unix
drwxrwxrwt 2 root root 4096 Nov 5 14:08 .ICE-unix
drwx------ 3 root root 4096 Nov 5 14:08 systemd-private-8420f88a9c25462da06b79691525babe-systemd-timesyncd.service-DndMOy
drwx------ 3 root root 4096 Nov 5 14:28 systemd-private-8420f88a9c25462da06b79691525babe-systemd-resolved.service-XnSMLg
drwxr-xr-x 23 root root 4096 Nov 5 14:28 ..
prwx------ 1 johnsi10 johnsi10 0 Nov 5 14:41 clr-debug-pipe-2085-194465-out
prwx------ 1 johnsi10 johnsi10 0 Nov 5 14:41 clr-debug-pipe-2085-194465-in
srw------- 1 johnsi10 johnsi10 0 Nov 5 14:41 dotnet-diagnostic-2085-194465-socket
-rw-r--r-- 1 johnsi10 johnsi10 0 Nov 5 17:00 i-updated-something.txt
drwxrwxrwt 9 root root 4096 Nov 5 17:00 .
What does a failure look like?
Let’s make the touch command fail:
strategy:
rolling:
maxParallel: 2
deploy:
steps:
- bash: |
#!/bin/bash
set -x
touch /sys/file
pwd
uname -a
displayName: 'touch a file'
While that failed, the step passed…
/bin/bash --noprofile --norc /home/johnsi10/azagent/_work/_temp/f8bbb55b-016b-4bf8-9a5e-f3ed0e75fe33.sh
+ touch /sys/file
touch: cannot touch '/sys/file': Permission denied
+ pwd
+ uname -a
/home/johnsi10/azagent/_work/1/s
Linux ijsamplelinuxvm 5.4.0-1031-azure #32~18.04.1-Ubuntu SMP Tue Oct 6 10:03:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
However, we can force a fail
- bash: |
#!/bin/bash
set -x
touch /sys/file
pwd
uname -a
exit 1
displayName: 'touch a file'
###
Deploying to a subset
We can also narrow the set to a subset based on tags. For instance, if i say just update the linode agent:
- deployment: vmupdater
displayName: "Update VMs"
environment:
name: DeploymentEnvironment
resourceType: VirtualMachine
tags: 'linode linux'
strategy:
Delivering build output
Lastly, say we want to deliver something we just built into the VM.. that’s fairly straightforward:
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
jobs:
- job: build
steps:
- script: echo Hello, world!
displayName: 'Run a one-line script'
- script: |
echo "My Test File" > $(Build.ArtifactStagingDirectory)/testfile.txt
displayName: 'Run a multi-line script'
- upload: $(Build.ArtifactStagingDirectory)
artifact: drop
- deployment: vmupdater
displayName: "Update VMs"
environment:
name: DeploymentEnvironment
resourceType: VirtualMachine
tags: 'linode linux'
strategy:
rolling:
maxParallel: 2
preDeploy:
steps:
- download: current
artifact: drop
deploy:
steps:
- bash: |
#!/bin/bash
set -x
export
pwd
ls -ltra $PIPELINE_WORKSPACE/drop
cp $PIPELINE_WORKSPACE/drop/testfile.txt /tmp
touch /tmp/test-file
uname -a
displayName: 'touch a file'
on:
failure:
steps:
- script: echo FAILED
success:
steps:
- script: echo PASSED
The key things are the upload of the “drop” artifact in the build stage and the download in the “preDeploy” stage in our “vmupdater” Deployment.
We can then see the results:
$ ssh root@173.255.198.161
Linux localhost 4.19.0-11-amd64 #1 SMP Debian 4.19.146-1 (2020-09-17) x86_64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Nov 5 16:56:03 2020 from 73.242.50.46
root@localhost:~# ls /tmp
clr-debug-pipe-844-55692-in
clr-debug-pipe-844-55692-out
dotnet-diagnostic-844-55692-socket
i-updated-something.txt
systemd-private-b6634aca9cfd431787a4b4e150baf88d-haveged.service-gAn5SW
systemd-private-b6634aca9cfd431787a4b4e150baf88d-systemd-timesyncd.service-KadW6I
test-file
testfile.txt
root@localhost:~# cat /tmp/testfile.txt
My Test File
Summary
I explored adding VMs from two cloud providers into a common pool. I then created a new YAML pipeline that would run commands on them. I showed how you can use tags to select a smaller set of the whole and then lastly covered how to transfer build output to the destination VMs.
