Published: May 11, 2023 by Isaac Johnson

I often run AWX in Kubernetes, either on-prem K3s, AKS or GKE. It’s easy to setup the Operator and instantiation. In fact, at work, one of our SREs even built out a very slick Terraform module just to sort out the Operator making adding AWX to our environments incredibly easy (but it wouldn’t be fair for me to share in-house work like that). But suffice to say, my hint is to use Terraform on Helm and Helm for an Operator.

I’ve been asked a lot lately about just VMs. What if we have very old AWXs, or a rotting Kubernetes, or a RedHat Tower (aka Ansible Automation Platform) instance to migrate? Can we just ‘put it on a VM’?

We’ll look at two approaches today: the single-node k3s, one with local Psql and one with Cloud SQL, then again with Docker Compose.

Installing via K3s single node

I’ll hop on an empty Linux box and and upgrade

Upgrade

$ sudo apt update && sudo apt -y upgrade

Now I’ll add k3s

$ curl -sfL https://get.k3s.io | sudo bash -
[INFO]  Finding release for channel stable
[INFO]  Using v1.26.4+k3s1 as release
[INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.26.4+k3s1/sha256sum-amd64.txt
[INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.26.4+k3s1/k3s
[INFO]  Verifying binary download
[INFO]  Installing k3s to /usr/local/bin/k3s
[INFO]  Skipping installation of SELinux RPM
[INFO]  Skipping /usr/local/bin/kubectl symlink to k3s, command exists in PATH at /snap/bin/kubectl
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Skipping /usr/local/bin/ctr symlink to k3s, command exists in PATH at /usr/bin/ctr
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO]  systemd: Starting k3s

$ sudo chmod 644 /etc/rancher/k3s/k3s.yaml

I’ll now verify it’s up

builder@builder-HP-EliteBook-745-G5:~$ cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
builder@builder-HP-EliteBook-745-G5:~$ kubectl get nodes
NAME                          STATUS   ROLES                  AGE     VERSION
builder-hp-elitebook-745-g5   Ready    control-plane,master   4m11s   v1.26.4+k3s1

Adding build-essential

$ sudo apt install git build-essential -y
Reading package lists... Done
Building dependency tree
Reading state information... Done
build-essential is already the newest version (12.8ubuntu1.1).
build-essential set to manually installed.
git is already the newest version (1:2.25.1-1ubuntu3.11).
git set to manually installed.
The following packages were automatically installed and are no longer required:
  docker-scan-plugin gir1.2-goa-1.0 libfprint-2-tod1 libfwupdplugin1 libllvm10 libxmlb1 linux-headers-5.15.0-70-generic linux-hwe-5.15-headers-5.15.0-70
  linux-image-5.15.0-70-generic linux-modules-5.15.0-70-generic linux-modules-extra-5.15.0-70-generic
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Now I can clone the AWX repo

$ git clone https://github.com/ansible/awx-operator.git
Cloning into 'awx-operator'...
remote: Enumerating objects: 8669, done.
remote: Counting objects: 100% (1342/1342), done.
remote: Compressing objects: 100% (170/170), done.
remote: Total 8669 (delta 1214), reused 1208 (delta 1171), pack-reused 7327
Receiving objects: 100% (8669/8669), 2.30 MiB | 6.90 MiB/s, done.
Resolving deltas: 100% (5026/5026), done.

$ cd awx-operator
$ sudo apt install curl jq -y
$ RELEASE_TAG=`curl -s https://api.github.com/repos/ansible/awx-operator/releases/latest | grep tag_name | cut -d '"' -f 4`
$ git checkout $RELEASE_TAG
Note: switching to '2.1.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 4fe482b Unpin ubuntu version for feature branch CI build (#1400) (#1401)

Now let’s make deploy

$ export NAMESPACE=awx
$ make deploy
Warning: resource namespaces/awx is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
namespace/awx configured
customresourcedefinition.apiextensions.k8s.io/awxbackups.awx.ansible.com created
customresourcedefinition.apiextensions.k8s.io/awxrestores.awx.ansible.com created
customresourcedefinition.apiextensions.k8s.io/awxs.awx.ansible.com created
serviceaccount/awx-operator-controller-manager created
role.rbac.authorization.k8s.io/awx-operator-awx-manager-role created
role.rbac.authorization.k8s.io/awx-operator-leader-election-role created
clusterrole.rbac.authorization.k8s.io/awx-operator-metrics-reader created
clusterrole.rbac.authorization.k8s.io/awx-operator-proxy-role created
rolebinding.rbac.authorization.k8s.io/awx-operator-awx-manager-rolebinding created
rolebinding.rbac.authorization.k8s.io/awx-operator-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/awx-operator-proxy-rolebinding created
configmap/awx-operator-awx-manager-config created
service/awx-operator-controller-manager-metrics-service created
deployment.apps/awx-operator-controller-manager created

I can now see it running

$ kubectl get pods
NAME                                              READY   STATUS    RESTARTS   AGE
awx-operator-controller-manager-cdb5fcc4d-fxnpl   2/2     Running   0          83s

Create a pvc

$ cat <<EOF | kubectl create -f -
> apiVersion: v1
> kind: PersistentVolumeClaim
> metadata:
>   name: static-data-pvc
>   namespace: awx
> spec:
>   accessModes:
>     - ReadWriteOnce
>   storageClassName: local-path
>   resources:
>     requests:
>       storage: 5Gi
> EOF
persistentvolumeclaim/static-data-pvc created

With the PVC created, we can add AWX using the Operator

$ cat <<EOF | kubectl create -f -
> apiVersion: awx.ansible.com/v1beta1
> kind: AWX
> metadata:
>   name: awx
> spec:
>   service_type: nodeport
>   projects_persistence: true
>   projects_storage_access_mode: ReadWriteOnce
>   web_extra_volume_mounts: |
>     - name: static-data
>       mountPath: /var/lib/projects
>   extra_volumes: |
>     - name: static-data
>       persistentVolumeClaim:
>         claimName: static-data-pvc
> EOF
awx.awx.ansible.com/awx created

Check on Pods

$ kubectl get pods -l "app.kubernetes.io/managed-by=awx-operator"
NAME                        READY   STATUS     RESTARTS   AGE
awx-postgres-13-0           1/1     Running    0          67s
awx-task-69cbbf589d-ts2kw   0/4     Init:0/2   0          26s

$ kubectl get pods -l "app.kubernetes.io/managed-by=awx-operator"
NAME                        READY   STATUS    RESTARTS   AGE
awx-postgres-13-0           1/1     Running   0          2m4s
awx-task-69cbbf589d-ts2kw   4/4     Running   0          83s
awx-web-595686bf8b-wwcxr    3/3     Running   0          11s

I’ll see what Port is used

$ kubectl get svc -l "app.kubernetes.io/managed-by=awx-operator"
NAME              TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
awx-postgres-13   ClusterIP   None           <none>        5432/TCP       4m20s
awx-service       NodePort    10.43.200.40   <none>        80:32471/TCP   3m42s

I can check the port

I can fetch the admin password

$ kubectl get secrets awx-admin-password -o json | jq -r .data.password | base64 --decode && echo
M0c3m8Dxikz8aX78uE1sMBxfAFgRLyd2

I need to create an R53 A record first

$ cat r53-awxvm.json
{
  "Comment": "CREATE awxvm fb.s A record ",
  "Changes": [
    {
      "Action": "CREATE",
      "ResourceRecordSet": {
        "Name": "awxvm.freshbrewed.science",
        "Type": "A",
        "TTL": 300,
        "ResourceRecords": [
          {
            "Value": "73.242.50.46"
          }
        ]
      }
    }
  ]
}

$ aws route53 change-resource-record-sets --hosted-zone-id Z39E8QFU0F9PZP --change-batch file://r53-awxvm.json
{
    "ChangeInfo": {
        "Id": "/change/C02213283GPL3VAKTIKAS",
        "Status": "PENDING",
        "SubmittedAt": "2023-05-10T01:07:56.273Z",
        "Comment": "CREATE awxvm fb.s A record "
    }
}

Then I can apply ingress

$ cat ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
  labels:
    app.kubernetes.io/instance: awxvmingress
  name: awxvmingress
spec:
  rules:
  - host: awxvm.freshbrewed.science
    http:
      paths:
      - backend:
          service:
            name: awx-external-ip
            port:
              number: 80
        path: /
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - awxvm.freshbrewed.science
    secretName: awxvm-tls
---
apiVersion: v1
kind: Service
metadata:
  name: awx-external-ip
spec:
  ports:
  - name: awxapp
    port: 80
    protocol: TCP
    targetPort: 32471
  clusterIP: None
  type: ClusterIP
---
apiVersion: v1
kind: Endpoints
metadata:
  name: awx-external-ip
subsets:
- addresses:
  - ip: 192.168.1.35
  ports:
  - name: awxapp
    port: 32471
    protocol: TCP

Now we can apply

$ kubectl apply -f ingress.yaml
ingress.networking.k8s.io/awxvmingress created
service/awx-external-ip created
endpoints/awx-external-ip created

and it works just fine

Cloud SQL

Let’s create a quick Cloud SQL PostgreSQL DB we can use for the backend

I’ll choose PostgreSQL

Then make a Development PG 14 Db

We can now see it being created

We can see the Public IP in Connections

I’ll now create a secret to provide the PSQL connection details

$ cat awx-db-secret.yaml
---
apiVersion: v1
kind: Secret
metadata:
  name: awx-postgres-configuration
  namespace: awx
stringData:
  host: "34.27.128.236"
  port: "5432"
  database: awx
  username: postgres
  password: myPassword
  sslmode: prefer
  type: unmanaged
type: Opaque


$ kubectl apply -f awx-db-secret.yaml
secret/awx-postgres-configuration created

Now I’ll recreate the AWX instance with it

$ kubectl delete awx awx
awx.awx.ansible.com "awx" deleted


$ cat awx.yaml
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  service_type: nodeport
  projects_persistence: true
  projects_storage_access_mode: ReadWriteOnce
  postgres_configuration_secret: awx-postgres-configuration
  web_extra_volume_mounts: |
    - name: static-data
      mountPath: /var/lib/projects
  extra_volumes: |
    - name: static-data
      persistentVolumeClaim:
        claimName: static-data-pvc

$ kubectl apply -f awx.yaml
awx.awx.ansible.com/awx created

We can check the pods and see it running, though this time without the postgres pod as we didn’t need the PostgreSQL database

$ kubectl get pods
NAME                                              READY   STATUS    RESTARTS   AGE
awx-operator-controller-manager-cdb5fcc4d-fxnpl   2/2     Running   0          129m
awx-task-7c496568bc-24g9b                         4/4     Running   0          15s
awx-web-68b4cb8ff-dsvjz                           3/3     Running   0          4s

This time the service is on 30265

$ kubectl get svc
NAME                                              TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
awx-operator-controller-manager-metrics-service   ClusterIP   10.43.37.242   <none>        8443/TCP       130m
awx-service                                       NodePort    10.43.150.20   <none>        80:30625/TCP   71s

I’ll update the Ingress Service port and Endpoint

$ cat AwxVM-Ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
  labels:
    app.kubernetes.io/instance: awxvmingress
  name: awxvmingress
spec:
  rules:
  - host: awxvm.freshbrewed.science
    http:
      paths:
      - backend:
          service:
            name: awx-external-ip
            port:
              number: 80
        path: /
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - awxvm.freshbrewed.science
    secretName: awxvm-tls
---
apiVersion: v1
kind: Service
metadata:
  name: awx-external-ip
spec:
  ports:
  - name: awxapp
    port: 80
    protocol: TCP
    targetPort: 30625
  clusterIP: None
  type: ClusterIP
---
apiVersion: v1
kind: Endpoints
metadata:
  name: awx-external-ip
subsets:
- addresses:
  - ip: 192.168.1.35
  ports:
  - name: awxapp
    port: 30625
    protocol: TCP

$ kubectl apply -f AwxVM-Ingress.yaml
ingress.networking.k8s.io/awxvmingress unchanged
service/awx-external-ip configured
endpoints/awx-external-ip configured

And we can see it is being used

via cli/python

First, I’ll add some libraries I believe we’ll need.

$ pip3 install sphinx sphinxcontrib-autoprogram
Requirement already satisfied: sphinx in /home/builder/.local/lib/python3.8/site-packages (7.0.0)
Requirement already satisfied: sphinxcontrib-autoprogram in /home/builder/.local/lib/python3.8/site-packages (0.1.8)
Requirement already satisfied: sphinxcontrib-applehelp in /home/builder/.local/lib/python3.8/site-packages (from sphinx) (1.0.4)
Requirement already satisfied: importlib-metadata>=4.8; python_version < "3.10" in /usr/local/lib/python3.8/dist-packages (from sphinx) (4.11.4)
Requirement already satisfied: sphinxcontrib-devhelp in /home/builder/.local/lib/python3.8/site-packages (from sphinx) (1.0.2)
Requirement already satisfied: sphinxcontrib-htmlhelp>=2.0.0 in /home/builder/.local/lib/python3.8/site-packages (from sphinx) (2.0.1)
Requirement already satisfied: packaging>=21.0 in /home/builder/.local/lib/python3.8/site-packages (from sphinx) (23.1)
Requirement already satisfied: Jinja2>=3.0 in /home/builder/.local/lib/python3.8/site-packages (from sphinx) (3.1.2)
Requirement already satisfied: sphinxcontrib-jsmath in /home/builder/.local/lib/python3.8/site-packages (from sphinx) (1.0.1)
Requirement already satisfied: Pygments>=2.13 in /home/builder/.local/lib/python3.8/site-packages (from sphinx) (2.15.1)
Requirement already satisfied: alabaster<0.8,>=0.7 in /home/builder/.local/lib/python3.8/site-packages (from sphinx) (0.7.13)
Requirement already satisfied: snowballstemmer>=2.0 in /home/builder/.local/lib/python3.8/site-packages (from sphinx) (2.2.0)
Requirement already satisfied: babel>=2.9 in /home/builder/.local/lib/python3.8/site-packages (from sphinx) (2.12.1)
Requirement already satisfied: requests>=2.25.0 in /usr/local/lib/python3.8/dist-packages (from sphinx) (2.28.0)
Requirement already satisfied: sphinxcontrib-qthelp in /home/builder/.local/lib/python3.8/site-packages (from sphinx) (1.0.3)
Requirement already satisfied: docutils<0.20,>=0.18.1 in /home/builder/.local/lib/python3.8/site-packages (from sphinx) (0.19)
Requirement already satisfied: sphinxcontrib-serializinghtml>=1.1.5 in /home/builder/.local/lib/python3.8/site-packages (from sphinx) (1.1.5)
Requirement already satisfied: imagesize>=1.3 in /home/builder/.local/lib/python3.8/site-packages (from sphinx) (1.4.1)
Requirement already satisfied: six in /usr/local/lib/python3.8/dist-packages (from sphinxcontrib-autoprogram) (1.16.0)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.8/dist-packages (from importlib-metadata>=4.8; python_version < "3.10"->sphinx) (3.8.0)
Requirement already satisfied: MarkupSafe>=2.0 in /home/builder/.local/lib/python3.8/site-packages (from Jinja2>=3.0->sphinx) (2.1.2)
Requirement already satisfied: pytz>=2015.7; python_version < "3.9" in /usr/local/lib/python3.8/dist-packages (from babel>=2.9->sphinx) (2022.1)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.8/dist-packages (from requests>=2.25.0->sphinx) (2.0.12)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests>=2.25.0->sphinx) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests>=2.25.0->sphinx) (1.26.9)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests>=2.25.0->sphinx) (2022.6.15)

Add Sphynx build

$ sudo apt install python3-sphinx
...
Setting up python3-sphinx (1.8.5-7ubuntu3) ...
update-alternatives: using /usr/share/sphinx/scripts/python3/sphinx-apidoc to provide /usr/bin/sphinx-apidoc (sphinx-apidoc) in auto mode
update-alternatives: using /usr/share/sphinx/scripts/python3/sphinx-autogen to provide /usr/bin/sphinx-autogen (sphinx-autogen) in auto mode
update-alternatives: using /usr/share/sphinx/scripts/python3/sphinx-build to provide /usr/bin/sphinx-build (sphinx-build) in auto mode
update-alternatives: using /usr/share/sphinx/scripts/python3/sphinx-quickstart to provide /usr/bin/sphinx-quickstart (sphinx-quickstart) in auto mode

Install awxkit

builder@builder-HP-EliteBook-745-G5:~/awx$ pip3 install -e ./awxkit
Obtaining file:///home/builder/awx/awxkit
Requirement already satisfied: PyYAML in /usr/local/lib/python3.8/dist-packages (from awxkit==22.2.1.dev1+g53260213ba) (6.0)
Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from awxkit==22.2.1.dev1+g53260213ba) (2.28.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.8/dist-packages (from requests->awxkit==22.2.1.dev1+g53260213ba) (2.0.12)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->awxkit==22.2.1.dev1+g53260213ba) (1.26.9)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->awxkit==22.2.1.dev1+g53260213ba) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->awxkit==22.2.1.dev1+g53260213ba) (2022.6.15)
Installing collected packages: awxkit
  Running setup.py develop for awxkit
Successfully installed awxkit

Then add local ansible

builder@builder-HP-EliteBook-745-G5:~/awx$ sudo apt-add-repository ppa:ansible/ansible
 Ansible is a radically simple IT automation platform that makes your applications and systems easier to deploy. Avoid writing scripts or custom code to deploy and update your applications— automate in a language that approaches plain English, using SSH, with no agents to install on remote systems.
...


builder@builder-HP-EliteBook-745-G5:~/awx$ sudo apt install ansible
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  docker-scan-plugin gir1.2-goa-1.0 libfprint-2-tod1 libfwupdplugin1 libllvm10 libxmlb1 linux-headers-5.15.0-69-generic linux-hwe-5.15-headers-5.15.0-69
  linux-image-5.15.0-69-generic linux-modules-5.15.0-69-generic linux-modules-extra-5.15.0-69-generic
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
...

In the AWX dir, we run make docker-compose-build

builder@builder-HP-EliteBook-745-G5:~/awx$ make docker-compose-build
Unable to import setuptools-scm, attempting to install now...
Traceback (most recent call last):
  File "tools/scripts/scm_version.py", line 7, in <module>
    from setuptools_scm import get_version
ModuleNotFoundError: No module named 'setuptools_scm'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tools/scripts/scm_version.py", line 12, in <module>
    subprocess.check_output([sys.executable, '-m', 'ensurepip'])
  File "/usr/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'ensurepip']' returned non-zero exit status 1.
ansible-playbook tools/ansible/dockerfile.yml \
        -e dockerfile_name=Dockerfile.dev \
        -e build_dev=True \
        -e receptor_image=quay.io/ansible/receptor:devel
/usr/lib/python3/dist-packages/paramiko/transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [Render AWX Dockerfile and sources] **************************************************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************************************************************
ok: [localhost]

TASK [dockerfile : Create _build directory] ***********************************************************************************************************************************
changed: [localhost]

TASK [dockerfile : Render supervisor configs] *********************************************************************************************************************************
changed: [localhost] => (item=supervisor_web.conf)
changed: [localhost] => (item=supervisor_task.conf)
changed: [localhost] => (item=supervisor_rsyslog.conf)

TASK [dockerfile : Render Dockerfile] *****************************************************************************************************************************************
changed: [localhost]

PLAY RECAP ********************************************************************************************************************************************************************
localhost                  : ok=4    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

DOCKER_BUILDKIT=1 docker build \
        -f Dockerfile.dev \
        -t ghcr.io/ansible/awx_devel:devel \
        --build-arg BUILDKIT_INLINE_CACHE=1 \
        --cache-from=ghcr.io/ansible/awx_devel:devel .
[+] Building 257.1s (49/49) FINISHED
...

It’s a pretty big image when done, we can see it in images

$ docker images
REPOSITORY                  TAG       IMAGE ID       CREATED          SIZE
ghcr.io/ansible/awx_devel   devel     43d7e88286bd   22 seconds ago   1.89GB
cloudcustodian/c7n          latest    610f126c34eb   10 months ago    895MB
hello-world                 latest    feb5d9fea6a5   19 months ago    13.3kB

Get the receptor image

builder@builder-HP-EliteBook-745-G5:~/awx$ docker pull quay.io/ansible/receptor
Using default tag: latest
latest: Pulling from ansible/receptor
250158a85877: Already exists
b109b11f9b12: Pull complete
ea6f0d6d06e6: Pull complete
7ebed36974ff: Pull complete
6dd101beeaef: Pull complete
4105dca5b1e7: Pull complete
Digest: sha256:b647b1156da587d7556f3ba179ff93171f4596b8878cd35ac591cc5ebb727fe7
Status: Downloaded newer image for quay.io/ansible/receptor:latest
quay.io/ansible/receptor:latest

Make Docker Compose for the remainder

builder@builder-HP-EliteBook-745-G5:~/awx$ make docker-compose
Unable to import setuptools-scm, attempting to install now...
Traceback (most recent call last):
  File "tools/scripts/scm_version.py", line 7, in <module>
    from setuptools_scm import get_version
ModuleNotFoundError: No module named 'setuptools_scm'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tools/scripts/scm_version.py", line 12, in <module>
    subprocess.check_output([sys.executable, '-m', 'ensurepip'])
  File "/usr/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'ensurepip']' returned non-zero exit status 1.
ansible-playbook -i tools/docker-compose/inventory tools/docker-compose/ansible/sources.yml \
    -e awx_image=ghcr.io/ansible/awx_devel \
    -e awx_image_tag=devel \
    -e receptor_image=quay.io/ansible/receptor:devel \
    -e control_plane_node_count=1 \
    -e execution_node_count=0 \
    -e minikube_container_group=false \
    -e enable_keycloak=false \
    -e enable_ldap=false \
    -e enable_splunk=false \
    -e enable_prometheus=false \
    -e enable_grafana=false \
    -e enable_tacacs=false \

/usr/lib/python3/dist-packages/paramiko/transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,

PLAY [Render AWX Dockerfile and sources] **************************************************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************************************************************
ok: [localhost]

TASK [sources : Create _sources directories] **********************************************************************************************************************************
changed: [localhost] => (item=secrets)
changed: [localhost] => (item=receptor)

TASK [sources : Detect secrets] ***********************************************************************************************************************************************
ok: [localhost] => (item=pg_password)
ok: [localhost] => (item=secret_key)
ok: [localhost] => (item=broadcast_websocket_secret)
ok: [localhost] => (item=admin_password)

TASK [sources : Generate secrets if needed] ***********************************************************************************************************************************
changed: [localhost] => (item=pg_password)
changed: [localhost] => (item=secret_key)
changed: [localhost] => (item=broadcast_websocket_secret)
changed: [localhost] => (item=admin_password)

TASK [sources : Include generated secrets unless they are explicitly passed in] ***********************************************************************************************
ok: [localhost] => (item=None)
ok: [localhost] => (item=None)
ok: [localhost] => (item=None)
ok: [localhost] => (item=None)
ok: [localhost]

TASK [sources : Write out SECRET_KEY] *****************************************************************************************************************************************
changed: [localhost]

TASK [sources : Render configuration templates] *******************************************************************************************************************************
changed: [localhost] => (item=database.py)
changed: [localhost] => (item=local_settings.py)
changed: [localhost] => (item=websocket_secret.py)
changed: [localhost] => (item=haproxy.cfg)
changed: [localhost] => (item=nginx.conf)
changed: [localhost] => (item=nginx.locations.conf)

TASK [sources : Get OS info for sdb] ******************************************************************************************************************************************
ok: [localhost]

TASK [sources : Get user UID] *************************************************************************************************************************************************
ok: [localhost]

TASK [sources : Set fact with user UID] ***************************************************************************************************************************************
ok: [localhost]

TASK [sources : Set global version if not provided] ***************************************************************************************************************************
skipping: [localhost]

TASK [sources : Generate Private RSA key for signing work] ********************************************************************************************************************
skipping: [localhost]

TASK [sources : Generate public RSA key for signing work] *********************************************************************************************************************
skipping: [localhost]

TASK [sources : Include LDAP tasks if enabled] ********************************************************************************************************************************
skipping: [localhost]

TASK [sources : Render Docker-Compose] ****************************************************************************************************************************************
changed: [localhost]

TASK [sources : Render Receptor Config(s) for Control Plane] ******************************************************************************************************************
changed: [localhost] => (item=1)

TASK [sources : Create Receptor Config Lock File] *****************************************************************************************************************************
changed: [localhost] => (item=1)

TASK [sources : Render Receptor Config(s) for Control Plane] ******************************************************************************************************************
ok: [localhost] => (item=1)

TASK [sources : Render Receptor Hop Config] ***********************************************************************************************************************************
skipping: [localhost]

TASK [sources : Render Receptor Worker Config(s)] *****************************************************************************************************************************
skipping: [localhost] => (item=1)

TASK [sources : Render prometheus config] *************************************************************************************************************************************
skipping: [localhost]

PLAY RECAP ********************************************************************************************************************************************************************
localhost                  : ok=14   changed=7    unreachable=0    failed=0    skipped=7    rescued=0    ignored=0

docker-compose -f tools/docker-compose/_sources/docker-compose.yml  up  --remove-orphans
make: docker-compose: Command not found
make: *** [Makefile:532: docker-compose] Error 127

I realized I had docker, but not docker-compose

builder@builder-HP-EliteBook-745-G5:~/awx$ which docker
/usr/bin/docker
builder@builder-HP-EliteBook-745-G5:~/awx$ which docker-compose
builder@builder-HP-EliteBook-745-G5:~/awx$

I’ll grab the latest release as of writing

builder@builder-HP-EliteBook-745-G5:~/awx$ sudo curl -L "https://github.com/docker/compose/releases/download/v2.17.3/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/
docker-compose
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 51.9M  100 51.9M    0     0  49.0M      0  0:00:01  0:00:01 --:--:-- 87.8M
builder@builder-HP-EliteBook-745-G5:~/awx$ sudo chmod +x /usr/local/bin/docker-compose
builder@builder-HP-EliteBook-745-G5:~/awx$ docker-compose --version
Docker Compose version v2.17.3

This time the make got farther

I had to stop and run again as something went wrong the first time.

When done, it got to a point it was listening

tools_awx_1       | awx-uwsgi stderr | mounting awx.wsgi:application on /
tools_awx_1       | awx-uwsgi stderr | mounting awx.wsgi:application on /
tools_awx_1       | awx-uwsgi stderr |
tools_awx_1       | awx-dispatcher stderr | 2023-05-10 11:51:15,187 WARNING  [-] awx.main.dispatch.periodic periodic beat started
tools_awx_1       | awx-dispatcher stderr |
tools_awx_1       | awx-dispatcher stderr | 2023-05-10 11:51:15,211 INFO     [-] awx.main.dispatch Running worker dispatcher listening to queues ['tower_broadcast_all', 'tower_settings_change', 'awx_1']
tools_awx_1       | awx-dispatcher stderr |
tools_awx_1       | awx-dispatcher stderr | 2023-05-10 11:51:15,287 WARNING  [-] awx.main.tasks.system Heartbeat skew - interval=70.7982, expected=60
tools_awx_1       | awx-dispatcher stderr |
tools_awx_1       | awx-daphne stderr | 2023-05-10 11:51:15,361 INFO     [-] daphne.cli Starting server at tcp:port=8051:interface=127.0.0.1
tools_awx_1       | awx-daphne stderr |
tools_awx_1       | awx-daphne stderr | 2023-05-10 11:51:15,361 INFO     Starting server at tcp:port=8051:interface=127.0.0.1
tools_awx_1       | awx-daphne stderr |
tools_awx_1       | awx-daphne stderr | 2023-05-10 11:51:15,365 INFO     [-] daphne.server HTTP/2 support not enabled (install the http2 and tls Twisted extras)
tools_awx_1       | awx-daphne stderr |
tools_awx_1       | awx-wsrelay stderr | 2023-05-10 11:51:15,366 INFO     [-] awx.main.wsrelay Active instance with hostname awx_1 is registered.
tools_awx_1       | awx-wsrelay stderr |
tools_awx_1       | awx-daphne stderr | 2023-05-10 11:51:15,365 INFO     HTTP/2 support not enabled (install the http2 and tls Twisted extras)
tools_awx_1       | awx-daphne stderr |
tools_awx_1       | awx-daphne stderr | 2023-05-10 11:51:15,367 INFO     [-] daphne.server Configuring endpoint tcp:port=8051:interface=127.0.0.1
tools_awx_1       | awx-daphne stderr | 2023-05-10 11:51:15,367 INFO     Configuring endpoint tcp:port=8051:interface=127.0.0.1
tools_awx_1       | awx-daphne stderr |
tools_awx_1       | awx-daphne stderr | 2023-05-10 11:51:15,368 INFO     [-] daphne.server Listening on TCP address 127.0.0.1:8051
tools_awx_1       | awx-daphne stderr | 2023-05-10 11:51:15,368 INFO     Listening on TCP address 127.0.0.1:8051
tools_awx_1       | awx-daphne stderr |
tools_awx_1       | 2023-05-10 11:51:15,566 INFO waiting for awx-rsyslogd to stop
tools_awx_1       | 2023-05-10 11:51:15,567 INFO stopped: awx-rsyslogd (exit status 0)
tools_awx_1       | 2023-05-10 11:51:15,573 INFO spawned: 'awx-rsyslogd' with pid 267
tools_awx_1       | 2023-05-10 11:51:16,577 INFO success: awx-rsyslogd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
tools_awx_1       | awx-uwsgi stderr | WSGI app 0 (mountpoint='/') ready in 2 seconds on interpreter 0xe26a80 pid: 255 (default app)
tools_awx_1       | awx-uwsgi stderr |

I initially saw some template code about node production, etc.

In another shell, I fired up a docker exec to build a new UI

builder@builder-HP-EliteBook-745-G5:~$ docker exec tools_awx_1 make clean-ui ui-devel
rm -rf node_modules
rm -rf awx/ui/node_modules
rm -rf awx/ui/build
rm -rf awx/ui/src/locales/_build
rm -rf awx/ui/.ui-built
mkdir -p awx/ui/build/static
NODE_OPTIONS=--max-old-space-size=6144 npm --prefix awx/ui --loglevel warn --force ci
npm WARN using --force Recommended protections disabled.
npm WARN deprecated source-map-url@0.4.1: See https://github.com/lydell/source-map-url#deprecated
...
Failed to parse source map from '/awx_devel/awx/ui/node_modules/rrule/src/weekday.ts' file: Error: ENOENT: no such file or directory, open '/awx_devel/awx/ui/node_modules/rrule/src/weekday.ts'

Search for the keywords to learn more about each warning.
To ignore, add // eslint-disable-next-line to the line before.

File sizes after gzip:

  908.96 kB  build/static/js/main.7a187b1d.js
  99.09 kB   build/static/css/main.e706ca3d.css
  61.87 kB   build/static/js/489.3819ec1a.chunk.js
  48.76 kB   build/static/js/118.95d6fd59.chunk.js
  45.7 kB    build/static/js/138.b38e12c9.chunk.js
  44.34 kB   build/static/js/787.f176f441.chunk.js
  44.07 kB   build/static/js/896.e61dac86.chunk.js
  43.21 kB   build/static/js/11.02c5663e.chunk.js
  42.29 kB   build/static/js/418.5971f1ef.chunk.js
  30.72 kB   build/static/js/311.86ad5143.chunk.js
  386 B      build/static/js/979.f16bcc0c.chunk.js

The bundle size is significantly larger than recommended.
Consider reducing it with code splitting: https://goo.gl/9VhYWB
You can also analyze the project dependencies: https://goo.gl/LeUzfb

The project was built assuming it is hosted at ./.
You can control this with the homepage field in your package.json.

The build folder is ready to be deployed.

Find out more about deployment here:

  https://cra.link/deployment

touch awx/ui/.ui-built
make[1]: Leaving directory '/awx_devel'

That took a long time to do, perhaps 15-20m to complete.

But when done, I could login to the host

Using the admin creds in the docker compose output

I got in

While this is nice, I would rather not be stuck in a shell to run AWX

I’ll ctrl-C and run the same command nohup

tools_redis_1     | 13:C 10 May 2023 12:05:50.013 * DB saved on disk
tools_redis_1     | 13:C 10 May 2023 12:05:50.013 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
tools_redis_1     | 1:M 10 May 2023 12:05:50.108 * Background saving terminated with success
^CGracefully stopping... (press Ctrl+C again to force)
Aborting on container exit...
[+] Running 3/3
 ✔ Container tools_awx_1       Stopped                                                                                              2.7s
 ✔ Container tools_postgres_1  Stopped                                                                                              0.3s
 ✔ Container tools_redis_1     Stopped                                                                                              0.4s
canceled
make: *** [Makefile:532: docker-compose] Error 130

This is so I can background it

builder@builder-HP-EliteBook-745-G5:~/awx$ cat ./dcawx.sh
#!/bin/bash
cd /home/builder/awx
make docker-compose
builder@builder-HP-EliteBook-745-G5:~/awx$ chmod 755 ./dcawx.sh

I can fire nohup now

builder@builder-HP-EliteBook-745-G5:~/awx$ nohup ./dcawx.sh &
[1] 36034
builder@builder-HP-EliteBook-745-G5:~/awx$ nohup: ignoring input and appending output to 'nohup.out'

builder@builder-HP-EliteBook-745-G5:~/awx$ tail -n5 nohup.out

TASK [sources : Render Receptor Config(s) for Control Plane] *******************
ok: [localhost] => (item=1)

TASK [sources : Create Receptor Config Lock File] ******************************

When it’s up, we can see it listening on /

builder@builder-HP-EliteBook-745-G5:~/awx$ tail -n5 nohup.out
tools_awx_1       | awx-uwsgi stderr |
tools_awx_1       | awx-uwsgi stderr | WSGI app 0 (mountpoint='/') ready in 3 seconds on interpreter 0x1805a80 pid: 264 (default app)
tools_awx_1       | awx-uwsgi stderr |
tools_awx_1       | awx-uwsgi stderr | WSGI app 0 (mountpoint='/') ready in 3 seconds on interpreter 0x1805a80 pid: 263 (default app)
tools_awx_1       | awx-uwsgi stderr |

and if I exit that shell, the docker compose stays running

We can see what is running using docker ps

$ docker ps
CONTAINER ID   IMAGE                             COMMAND                  CREATED         STATUS         PORTS


                                                                   NAMES
4a602345965e   ghcr.io/ansible/awx_devel:devel   "/entrypoint.sh laun…"   8 minutes ago   Up 4 minutes   0.0.0.0:2222->2222/tcp, :::2222->2222/tcp, 0.0.0.0:6899->6899/tcp, :::6899->6899/tcp, 0.0.0.0:7899-7999->7899-7999/tcp, :::7899-7999->7899-7999/tcp, 0.0.0.0:8013->8013/tcp, :::8013->8013/tcp, 0.0.0.0:8043->8043/tcp, :::8043->8043/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 22/tcp, 0.0.0.0:8888->8888/tcp, :::8888->8888/tcp, 0.0.0.0:3000->3001/tcp, :::3000->3001/tcp   tools_awx_1
17511079a225   postgres:12                       "docker-entrypoint.s…"   8 minutes ago   Up 4 minutes   5432/tcp

Summary

This was a quick example guide. I still prefer to run AWX in Kubernetes as it’s really designed to have a few inter-related services and containerized runners.

This was done mostly just to answer the question: How can we run AWX on a VM (or in my case, a Linux laptop).