A topic that has come up a few times lately, in regards to YAML pipelines in Azure DevOps, is how to use Environments for non-containerized workloads.  Most of to whom I speak are pretty comfortable with the Kubernetes "environment" with namespaces and gates, but how do the "VirtualMachines" work in the context of deployment environments?

Let's take a moment to clear up some confusion on this topic and do some working examples.

Creating a new Environment

Create with the Resource of Virtual Machines

You can also add "Virtual machines" to an existing Environment
Only Windows and Linux supported as of writing

The command to copy shows:

mkdir azagent;cd azagent;curl -fkSL -o vstsagent.tar.gz https://vstsagentpackage.azureedge.net/agent/2.175.2/vsts-agent-linux-x64-2.175.2.tar.gz;tar -zxvf vstsagent.tar.gz; if [ -x "$(command -v systemctl)" ]; then ./config.sh --environment --environmentname "DeploymentEnvironment" --acceptteeeula --agent $HOSTNAME --url https://princessking.visualstudio.com/ --work _work --projectname 'Fortran_CICD' --auth PAT --token *************************************** --runasservice; sudo ./svc.sh install; sudo ./svc.sh start; else ./config.sh --environment --environmentname "DeploymentEnvironment" --acceptteeeula --agent $HOSTNAME --url https://princessking.visualstudio.com/ --work _work --projectname 'Fortran_CICD' --auth PAT --token ***************************************; ./run.sh; fi

It should be noted, the command DOES include the PAT - i am just masking here.

Create some VMs

We use can the Azure CLI to create one of the VMs

$ az vm create --name ijsamplelinuxvm --resource-group idjaksdemo --image UbuntuLTS --ssh-key-values @/Users/johnsi10/.ssh/id_rsa.pub
{- Finished ..
  "fqdns": "",
  "id": "/subscriptions/asdfasd-asdfasdfasd-asdfasdf-asdfasdf/resourceGroups/idjaksdemo/providers/Microsoft.Compute/virtualMachines/ijsamplelinuxvm",
  "location": "eastus",
  "macAddress": "00-0D-3A-12-C8-4A",
  "powerState": "VM running",
  "privateIpAddress": "10.0.0.4",
  "publicIpAddress": "52.170.222.194",
  "resourceGroup": "idjaksdemo",
  "zones": ""
}

Verify we can login

$ ssh 52.170.222.194
The authenticity of host '52.170.222.194 (52.170.222.194)' can't be established.
ECDSA key fingerprint is SHA256:w+sSHT0n4ZxLb0MSUkQe5grN8CDxXkk63/WK7dsEx7k.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '52.170.222.194' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 5.4.0-1031-azure x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Thu Nov  5 14:37:44 UTC 2020

  System load:  0.0               Processes:           109
  Usage of /:   4.4% of 28.90GB   Users logged in:     0
  Memory usage: 5%                IP address for eth0: 10.0.0.4
  Swap usage:   0%

0 packages can be updated.
0 updates are security updates.



The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

johnsi10@ijsamplelinuxvm:~$ sudo echo hi
hi

Install the agent

johnsi10@ijsamplelinuxvm:~$ mkdir azagent;cd azagent;curl -fkSL -o vstsagent.tar.gz https://vstsagentpackage.azureedge.net/agent/2.175.2/vsts-agent-linux-x64-2.175.2.tar.gz;tar -zxvf vstsagent.tar.gz; if [ -x "$(command -v systemctl)" ]; then ./config.sh --environment --environmentname "DeploymentEnvironment" --acceptteeeula --agent $HOSTNAME --url https://princessking.visualstudio.com/ --work _work --projectname 'Fortran_CICD' --auth PAT --token asdfasdfasdfasdfasdfasdfasdfasdfasdfasdf --runasservice; sudo ./svc.sh install; sudo ./svc.sh start; else ./config.sh --environment --environmentname "DeploymentEnvironment" --acceptteeeula --agent $HOSTNAME --url https://princessking.visualstudio.com/ --work _work --projectname 'Fortran_CICD' --auth PAT --token asdfasdfasdfasdfasdfasdfasdfasdfasdfasdf; ./run.sh; fi
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  130M  100  130M    0     0   183M      0 --:--:-- --:--:-- --:--:--  183M
./
./config.sh
./bin/
./bin/Agent.Listener.deps.json
./bin/System.IO.FileSystem.Watcher.dll
….
./externals/vso-task-lib/node_modules/.bin/shjs.cmd

  ___                      ______ _            _ _
 / _ \                     | ___ (_)          | (_)
/ /_\ \_____   _ _ __ ___  | |_/ /_ _ __   ___| |_ _ __   ___  ___
|  _  |_  / | | | '__/ _ \ |  __/| | '_ \ / _ \ | | '_ \ / _ \/ __|
| | | |/ /| |_| | | |  __/ | |   | | |_) |  __/ | | | | |  __/\__ \
\_| |_/___|\__,_|_|  \___| \_|   |_| .__/ \___|_|_|_| |_|\___||___/
                                   | |
        agent v2.175.2             |_|          (commit 5c4925c)


>> End User License Agreements:

Building sources from a TFVC repository requires accepting the Team Explorer Everywhere End User License Agreement. This step is not required for building sources from Git repositories.

A copy of the Team Explorer Everywhere license agreement can be found at:
  /home/johnsi10/azagent/externals/tee/license.html


>> Connect:

Connecting to server ...

>> Register Agent:

Scanning for tool capabilities.
Connecting to the server.
Enter Environment Virtual Machine resource tags? (Y/N) (press enter for N) > Y      
Enter Comma separated list of tags (e.g web, db) > azurevm linux
Successfully added the agent
Testing agent connection.
2020-11-05 14:41:03Z: Settings Saved.
Creating launch agent in /etc/systemd/system/vsts.agent.princessking..ijsamplelinuxvm.service
Run as user: johnsi10
Run as uid: 1000
gid: 1000
Created symlink /etc/systemd/system/multi-user.target.wants/vsts.agent.princessking..ijsamplelinuxvm.service → /etc/systemd/system/vsts.agent.princessking..ijsamplelinuxvm.service.

/etc/systemd/system/vsts.agent.princessking..ijsamplelinuxvm.service
● vsts.agent.princessking..ijsamplelinuxvm.service - Azure Pipelines Agent (princessking..ijsamplelinuxvm)
   Loaded: loaded (/etc/systemd/system/vsts.agent.princessking..ijsamplelinuxvm.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2020-11-05 14:41:06 UTC; 18ms ago
 Main PID: 2064 (runsvc.sh)
    Tasks: 2 (limit: 4075)
   CGroup: /system.slice/vsts.agent.princessking..ijsamplelinuxvm.service
           └─2064 /bin/bash /home/johnsi10/azagent/runsvc.sh

Nov 05 14:41:06 ijsamplelinuxvm systemd[1]: Started Azure Pipelines Agent (princessking..ijsamplelinuxvm).

We now see it listed

If we choose manage tags

We can see that tag we added:

Let’s also create a VM somewhere else as well:

Once logged in, create a user

root@localhost:~# adduser azdoagent
Adding user `azdoagent' ...
Adding new group `azdoagent' (1000) ...
Adding new user `azdoagent' (1000) with group `azdoagent' ...
Creating home directory `/home/azdoagent' ...
Copying files from `/etc/skel' ...
New password: 
Retype new password: 
passwd: password updated successfully
Changing the user information for azdoagent
Enter the new value, or press ENTER for the default
	Full Name []: AzDO user
	Room Number []: 
	Work Phone []: 
	Home Phone []: 
	Other []: 
Is the information correct? [Y/n] Y
root@localhost:~# usermod -aG sudo azdoagent

Then update sudoers sudo visudo

# Allow members of group sudo to execute any command
%sudo   ALL=(ALL:ALL) NOPASSWD:ALL

Validate

root@localhost:~# su - azdoagent
azdoagent@localhost:~$ sudo echo hi
hi

Adding that agent, we’ll use other tags

….

Scanning for tool capabilities.
Connecting to the server.
Enter Environment Virtual Machine resource tags? (Y/N) (press enter for N) > Y
Enter Comma separated list of tags (e.g web, db) > linode linux
Successfully added the agent
Testing agent connection.
2020-11-05 14:56:48Z: Settings Saved.
Creating launch agent in /etc/systemd/system/vsts.agent.princessking..localhost.service
Run as user: azdoagent
Run as uid: 1000
gid: 1000
Created symlink /etc/systemd/system/multi-user.target.wants/vsts.agent.princessking..localhost.service → /etc/systemd/system/vsts.agent.princessking..localhost.service.

/etc/systemd/system/vsts.agent.princessking..localhost.service
● vsts.agent.princessking..localhost.service - Azure Pipelines Agent (princessking..localhost)
   Loaded: loaded (/etc/systemd/system/vsts.agent.princessking..localhost.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2020-11-05 14:56:48 UTC; 11ms ago
 Main PID: 835 (runsvc.sh)
    Tasks: 2 (limit: 1149)
   Memory: 476.0K
   CGroup: /system.slice/vsts.agent.princessking..localhost.service
           ├─835 /bin/bash /home/azdoagent/azagent/runsvc.sh
           └─837 /bin/bash /home/azdoagent/azagent/runsvc.sh

Nov 05 14:56:48 localhost systemd[1]: Started Azure Pipelines Agent (princessking..localhost).

The only issue is that that host just knew itself as “localhost” .. but we can use the tags to see that indeed, it is the nanode

I will note that i tried to add my Mac, but it seems the binaries are not darwin (mac os) compatible

./config.sh: line 85: ./bin/Agent.Listener: cannot execute binary file
./run.sh: line 43: /Users/johnsi10/agentdemo/azagent/bin/Agent.Listener: cannot execute binary file

Using the VM Deployment

Lets create a new repo for our pipeline

Then we can start creating our pipeline by clicking setup build

We can do a simple starter pipeline

There is the basic one:

# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
 
trigger:
- master
 
pool:
 vmImage: 'ubuntu-latest'
 
steps:
- script: echo Hello, world!
 displayName: 'Run a one-line script'
 
- script: |
   echo Add other tasks to build, test, and deploy your project.
   echo See https://aka.ms/yaml
 displayName: 'Run a multi-line script'

But we can easily add our Deployment environment:

# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
 
trigger:
- master
 
pool:
 vmImage: 'ubuntu-latest'
 
jobs:
- job: build
 steps:
 - script: echo Hello, world!
   displayName: 'Run a one-line script'
 
 - script: |
     echo Add other tasks to build, test, and deploy your project.
     echo See https://aka.ms/yaml
   displayName: 'Run a multi-line script'
 
- deployment: vmupdater
 displayName: "Update VMs"
 environment:
    name: DeploymentEnvironment
    resourceType: VirtualMachine
 strategy:
    rolling:
      maxParallel: 2
      deploy:
        steps:
        - bash: |
            #!/bin/bash
            set -x
            touch /tmp/i-updated-something.txt
            pwd
            uname -a
          displayName: 'touch a file'
      on:
        failure:
           steps:
           - script: echo FAILED
        success:
           steps:
           - script: echo PASSED

and the output

Let’s look at some of the output on the “Deploy” stages

Linux ijsamplelinuxvm 5.4.0-1031-azure #32~18.04.1-Ubuntu SMP Tue Oct 6 10:03:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Linux localhost 4.19.0-11-amd64 #1 SMP Debian 4.19.146-1 (2020-09-17) x86_64 GNU/Linux

Lastly, if we hop on the hosts, we can see that the file was updated:

johnsi10@ijsamplelinuxvm:~$ ls -ltra /tmp
total 36
drwxrwxrwt  2 root     root     4096 Nov  5 14:08 .font-unix
drwxrwxrwt  2 root     root     4096 Nov  5 14:08 .XIM-unix
drwxrwxrwt  2 root     root     4096 Nov  5 14:08 .X11-unix
drwxrwxrwt  2 root     root     4096 Nov  5 14:08 .Test-unix
drwxrwxrwt  2 root     root     4096 Nov  5 14:08 .ICE-unix
drwx------  3 root     root     4096 Nov  5 14:08 systemd-private-8420f88a9c25462da06b79691525babe-systemd-timesyncd.service-DndMOy
drwx------  3 root     root     4096 Nov  5 14:28 systemd-private-8420f88a9c25462da06b79691525babe-systemd-resolved.service-XnSMLg
drwxr-xr-x 23 root     root     4096 Nov  5 14:28 ..
prwx------  1 johnsi10 johnsi10    0 Nov  5 14:41 clr-debug-pipe-2085-194465-out
prwx------  1 johnsi10 johnsi10    0 Nov  5 14:41 clr-debug-pipe-2085-194465-in
srw-------  1 johnsi10 johnsi10    0 Nov  5 14:41 dotnet-diagnostic-2085-194465-socket
-rw-r--r--  1 johnsi10 johnsi10    0 Nov  5 17:00 i-updated-something.txt
drwxrwxrwt  9 root     root     4096 Nov  5 17:00 .

What does a failure look like?

Let’s make the touch command fail:

 strategy:
    rolling:
      maxParallel: 2
      deploy:
        steps:
        - bash: |
            #!/bin/bash
            set -x
            touch /sys/file
            pwd
            uname -a
          displayName: 'touch a file'

While that failed, the step passed…

/bin/bash --noprofile --norc /home/johnsi10/azagent/_work/_temp/f8bbb55b-016b-4bf8-9a5e-f3ed0e75fe33.sh

+ touch /sys/file

touch: cannot touch '/sys/file': Permission denied

+ pwd

+ uname -a

/home/johnsi10/azagent/_work/1/s

Linux ijsamplelinuxvm 5.4.0-1031-azure #32~18.04.1-Ubuntu SMP Tue Oct 6 10:03:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

However, we can force a fail 


        - bash: |
            #!/bin/bash
            set -x
            touch /sys/file
            pwd
            uname -a
            exit 1
          displayName: 'touch a file'


Deploying to a subset

We can also narrow the set to a subset based on tags.  For instance, if i say just update the linode agent:

- deployment: vmupdater
 displayName: "Update VMs"
 environment:
    name: DeploymentEnvironment
    resourceType: VirtualMachine
    tags: 'linode linux'
 strategy:

Delivering build output

Lastly, say we want to deliver something we just built into the VM.. that’s fairly straightforward:


# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
 
trigger:
- master
 
pool:
 vmImage: 'ubuntu-latest'
 
jobs:
- job: build
 steps:
 - script: echo Hello, world!
   displayName: 'Run a one-line script'
 
 - script: |
     echo "My Test File" > $(Build.ArtifactStagingDirectory)/testfile.txt
   displayName: 'Run a multi-line script'
 
 - upload: $(Build.ArtifactStagingDirectory)
   artifact: drop
 
- deployment: vmupdater
 displayName: "Update VMs"
 environment:
    name: DeploymentEnvironment
    resourceType: VirtualMachine
    tags: 'linode linux'
 strategy:
    rolling:
      maxParallel: 2
      preDeploy:
        steps:
        - download: current
          artifact: drop
      deploy:
        steps:
        - bash: |
            #!/bin/bash
            set -x
            export
            pwd
            ls -ltra $PIPELINE_WORKSPACE/drop
            cp $PIPELINE_WORKSPACE/drop/testfile.txt /tmp
            touch /tmp/test-file
           
            uname -a
          displayName: 'touch a file'
      on:
        failure:
           steps:
           - script: echo FAILED
        success:
           steps:
           - script: echo PASSED

The key things are the upload of the “drop” artifact in the build stage and the download in the “preDeploy” stage in our “vmupdater” Deployment.

We can then see the results:

$ ssh root@173.255.198.161
Linux localhost 4.19.0-11-amd64 #1 SMP Debian 4.19.146-1 (2020-09-17) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Nov  5 16:56:03 2020 from 73.242.50.46
root@localhost:~# ls /tmp
clr-debug-pipe-844-55692-in
clr-debug-pipe-844-55692-out
dotnet-diagnostic-844-55692-socket
i-updated-something.txt
systemd-private-b6634aca9cfd431787a4b4e150baf88d-haveged.service-gAn5SW
systemd-private-b6634aca9cfd431787a4b4e150baf88d-systemd-timesyncd.service-KadW6I
test-file
testfile.txt
root@localhost:~# cat /tmp/testfile.txt 
My Test File

Summary

I explored adding VMs from two cloud providers into a common pool. I then created a new YAML pipeline that would run commands on them.  I showed how you can use tags to select a smaller set of the whole and then lastly covered how to transfer build output to the destination VMs.