Ubuntu Multipass - Better than Docker?
Multipass is billed as a “mini-cloud” for Mac and Windows machines. . It’s ubuntu’s play at a small virtual machine environment akin to Docker. How well does it perform and can we run Rancher’s k3s or perhaps even a full Kubernetes on it?
Installing
Download the binary from https://multipass.run/#install
In the case of macOS, it was a quick pkg install.
Running the basic “launch an instance” command took a bit (longer than docker)
I immediately tried to run commands in it, and also checked to see if it was really just running docker (does not appear to be).
JOHNSI10-M1:~ johnsi10$ multipass exec foo -- uname -a
Linux foo 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
JOHNSI10-M1:~ johnsi10$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
JOHNSI10-M1:~ johnsi10$
Getting a fresh shell was easy:
JOHNSI10-M1:~ johnsi10$ multipass shell
Launched: primary
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-65-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Sat Oct 5 10:04:13 CDT 2019
System load: 0.45 Processes: 115
Usage of /: 20.8% of 4.67GB Users logged in: 0
Memory usage: 13% IP address for enp0s2: 192.168.64.3
Swap usage: 0%
0 packages can be updated.
0 updates are security updates.
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
multipass@primary:~$
However, doing so launched a fresh instance called primary (doing it again re-used primary so it must be a default name)
$ multipass list
Name State IPv4 Image
primary Running 192.168.64.3 Ubuntu 18.04 LTS
foo Running 192.168.64.2 Ubuntu 18.04 LTS
The instances seem very fast.. I checked and they allocate about a Gb of memory by default:
multipass@primary:~$ free -m
total used free shared buff/cache available
Mem: 986 72 526 0 387 772
Swap: 0 0 0
multipass@primary:~$ cat /proc/meminfo
MemTotal: 1009716 kB
MemFree: 538700 kB
I thought it was cute under the ~root .ssh/authorized_keys they clearly don’t want you using root for things:
no-port-forwarding,no-agent-forwarding,no-X11-forwarding,command="echo 'Please login as the user \"multipass\" rather than the user \"root\".';echo;sleep 10" ssh-rsa A…
I added my ~/.ssh/id_rsa.pub to the end of ~root/.ssh/authorized_keys and tested root access with ssh…
JOHNSI10-M1:~ johnsi10$ multipass list
Name State IPv4 Image
primary Running 192.168.64.3 Ubuntu 18.04 LTS
foo Running 192.168.64.2 Ubuntu 18.04 LTS
JOHNSI10-M1:~ johnsi10$ ssh root@192.168.64.3
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-65-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Sat Oct 5 10:11:50 CDT 2019
System load: 0.07 Processes: 102
Usage of /: 23.9% of 4.67GB Users logged in: 0
Memory usage: 12% IP address for enp0s2: 192.168.64.3
Swap usage: 0%
0 packages can be updated.
0 updates are security updates.
Last login: Sat Oct 5 10:11:40 2019 from 192.168.64.1
Installing k3s
Let’s try getting k3s on there:
multipass@primary:~$ sudo su -
root@primary:~# curl -sfL https://get.k3s.io | sh -
[INFO] Finding latest release
[INFO] Using v0.9.1 as release
[INFO] Downloading hash https://github.com/rancher/k3s/releases/download/v0.9.1/sha256sum-amd64.txt
[INFO] Downloading binary https://github.com/rancher/k3s/releases/download/v0.9.1/k3s
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] Creating /usr/local/bin/kubectl symlink to k3s
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Creating /usr/local/bin/ctr symlink to k3s
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s.service
[INFO] systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO] systemd: Starting k3s
root@primary:~# sudo kubectl get nodes
NAME STATUS ROLES AGE VERSION
primary Ready master 24s v1.15.4-k3s.1
Can we increase (scale) our cluster?
root@primary:~# cat /var/lib/rancher/k3s/server/node-token
K10963f9c0e7f6c56556ca580f91d880a6a649c17a0b29b4df4ad3d0cba5e8ec180::node:5401832f9196e979e457e6294003d0ef
Adding a new node…
root@foo:~# curl -sfL https://get.k3s.io | K3S_URL=https://192.168.64.3:6443 K3S_TOKEN=`cat token` sh -
[INFO] Finding latest release
[INFO] Using v0.9.1 as release
[INFO] Downloading hash https://github.com/rancher/k3s/releases/download/v0.9.1/sha256sum-amd64.txt
[INFO] Downloading binary https://github.com/rancher/k3s/releases/download/v0.9.1/k3s
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] Creating /usr/local/bin/kubectl symlink to k3s
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Creating /usr/local/bin/ctr symlink to k3s
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-agent-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s-agent.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s-agent.service
[INFO] systemd: Enabling k3s-agent unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s-agent.service → /etc/systemd/system/k3s-agent.service.
[INFO] systemd: Starting k3s-agent
It worked!
multipass@primary:~$ sudo kubectl get nodes
NAME STATUS ROLES AGE VERSION
primary Ready master 31m v1.15.4-k3s.1
foo Ready worker 43s v1.15.4-k3s.1
Let’s get the kubeconfig
multipass@primary:~$ sudo cat /etc/rancher/k3s/k3s.yaml
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJWakNCL3FBREFnRUNBZ0VBTUFvR0NDcUdTTTQ5QkFNQ01DTXhJVEFmQmdOVkJBTU1HR3N6Y3kxelpYSjIKWlhJdFkyRkFNVFUzTURJNE9Ea3hOVEFlRncweE9URXdNRFV4TlRJeE5UVmFGdzB5T1RFd01ESXhOVEl4TlRWYQpNQ014SVRBZkJnTlZCQU1NR0dzemN5MXpaWEoyWlhJdFkyRkFNVFUzTURJNE9Ea3hOVEJaTUJNR0J5cUdTTTQ5CkFnRUdDQ3FHU000OUF3RUhBMElBQkw3VnhxY2VJL1dyVFRyZmhxeUxWMDVZaXBzNlJseGxXNHNXN3dNTEhjdy8KYUJjbHl3ZTdzdmxtbDZOZG1JSmF3MUgzS0d6NkZ3NVMzL3pPSU5TSkU0T2pJekFoTUE0R0ExVWREd0VCL3dRRQpBd0lDcERBUEJnTlZIUk1CQWY4RUJUQURBUUgvTUFvR0NDcUdTTTQ5QkFNQ0EwY0FNRVFDSUEzUkl3MmhQS05zCktoU1RYaFFrZTdXZVo5U0VTZEhZRjdUbklEQ3pwOFdzQWlBend4RU5HeFlvV2Q2T2FmbzBBUUZRRFowcE4xQTgKS3pCSmQzZ2Jnb1BRS0E9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
server: https://127.0.0.1:6443
name: default
contexts:
- context:
cluster: default
user: default
name: default
current-context: default
kind: Config
preferences: {}
users:
- name: default
user:
password: e2fbe383931c1b5f9500219a1272b570
username: admin
We can save that locally and just change the “server” line to match our master node:
JOHNSI10-M1:~ johnsi10$ sed -i.bak -e 's/127.0.0.1:6443/192.168.64.3/g' myk3sconfig
JOHNSI10-M1:~ johnsi10$ !cat
cat myk3sconfig | grep server
server: https://192.168.64.3
Now we can test locally:
JOHNSI10-M1:~ johnsi10$ export KUBECONFIG=$(pwd)/myk3sconfig
JOHNSI10-M1:~ johnsi10$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
foo Ready worker 4m18s v1.15.4-k3s.1
primary Ready master 35m v1.15.4-k3s.1
However, we don’t have any resources.. Which seems a bit odd:
JOHNSI10-M1:~ johnsi10$ kubectl api-resources --insecure-skip-tls-verify
NAME SHORTNAMES APIGROUP NAMESPACED KIND
JOHNSI10-M1:~ johnsi10$
It is working, however, with k3s:
multipass@foo2:~$ sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
multipass@foo2:~$ sudo chmod 644 ~/.kube/config
multipass@foo2:~$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-66f496764-s5bgd 1/1 Running 0 15m
kube-system helm-install-traefik-m6jlr 0/1 Completed 0 15m
kube-system svclb-traefik-5z86d 3/3 Running 0 15m
kube-system traefik-d869575c8-6pts7 1/1 Running 0 15m
When trying this way - using with sudo ./k3s server & from foo2, it worked fine. . I just used the same kubeconfig
JOHNSI10-M1:~ johnsi10$ vi ~/.kube/config
JOHNSI10-M1:~ johnsi10$ cat ~/.kube/config
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJWekNCL3FBREFnRUNBZ0VBTUFvR0NDcUdTTTQ5QkFNQ01DTXhJVEFmQmdOVkJBTU1HR3N6Y3kxelpYSjIKWlhJdFkyRkFNVFUzTURJNU16Z3hNREFlRncweE9URXdNRFV4TmpRek16QmFGdzB5T1RFd01ESXhOalF6TXpCYQpNQ014SVRBZkJnTlZCQU1NR0dzemN5MXpaWEoyWlhJdFkyRkFNVFUzTURJNU16Z3hNREJaTUJNR0J5cUdTTTQ5CkFnRUdDQ3FHU000OUF3RUhBMElBQkVNbGp4YzVXdnNDdHlUSEpOWDZTV3A3NEE1QnhkWWNQSjhSZjI0MjM0SW0KSUFQaGkrYjNrRzl0aDVEWjlFZ05xVjJlUDZ4WGRmR2IrSHFFU0Qrck55YWpJekFoTUE0R0ExVWREd0VCL3dRRQpBd0lDcERBUEJnTlZIUk1CQWY4RUJUQURBUUgvTUFvR0NDcUdTTTQ5QkFNQ0EwZ0FNRVVDSUE0U0FKbDBkbHdPCk1raFdQVTVqZUxqbHFsVGtUT3I5akpaV2RxNXdOQS9NQWlFQTh0b2hiWDFXdjA3ZEN4clIzME9aVy9RMytxV28Kd0FFMmd2ckVESk54QU1FPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
server: https://192.168.64.4:6443
name: default
contexts:
- context:
cluster: default
user: default
name: default
current-context: default
kind: Config
preferences: {}
users:
- name: default
user:
password: 235c8033da3e36820d379b2c0d3011d3
username: admin
JOHNSI10-M1:~ johnsi10$ unset KUBECONFIG
JOHNSI10-M1:~ johnsi10$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-66f496764-s5bgd 1/1 Running 0 19m
kube-system helm-install-traefik-m6jlr 0/1 Completed 0 19m
kube-system svclb-traefik-5z86d 3/3 Running 0 18m
kube-system traefik-d869575c8-6pts7 1/1 Running 0 18m
JOHNSI10-M1:~ johnsi10$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
foo2 Ready master 19m v1.15.4-k3s.1
Let’s try adding that foo node again:
root@foo:~# curl -sfL https://get.k3s.io | K3S_URL=https://192.168.64.4:6443 K3S_TOKEN=K109b57bbfe0e3251481743ed00c79a323e9792eababc18ff7505a7e10f84c8e6cf::node:c9416ab5ca8157b919e331b1a4a0878d sh -
[INFO] Finding latest release
[INFO] Using v0.9.1 as release
[INFO] Downloading hash https://github.com/rancher/k3s/releases/download/v0.9.1/sha256sum-amd64.txt
[INFO] Skipping binary downloaded, installed k3s matches hash
[INFO] Skipping /usr/local/bin/kubectl symlink to k3s, already exists
[INFO] Skipping /usr/local/bin/crictl symlink to k3s, already exists
[INFO] Skipping /usr/local/bin/ctr symlink to k3s, already exists
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-agent-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s-agent.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s-agent.service
[INFO] systemd: Enabling k3s-agent unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s-agent.service → /etc/systemd/system/k3s-agent.service.
[INFO] systemd: Starting k3s-agent
That worked!
JOHNSI10-M1:Documents johnsi10$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
foo2 Ready master 23m v1.15.4-k3s.1
foo Ready worker 27s v1.15.4-k3s.1
Testing with SonarQube chart
Let’s fire up Tiller then SQube..
First we need to get tiller working with RBAC:
JOHNSI10-M1:Documents johnsi10$ cat rbac-config.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
JOHNSI10-M1:Documents johnsi10$ kubectl create -f rbac-config.yaml
serviceaccount/tiller created
clusterrolebinding.rbac.authorization.k8s.io/tiller created
JOHNSI10-M1:Documents johnsi10$ helm init --service-account tiller --history-max 200
$HELM_HOME has been configured at /Users/johnsi10/.helm.
Warning: Tiller is already installed in the cluster.
(Use --client-only to suppress this message, or --upgrade to upgrade Tiller to the current version.)
Then install the chart
$ helm install stable/sonarqube
NAME: eating-bobcat
LAST DEPLOYED: Sat Oct 5 13:35:26 2019
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/ConfigMap
NAME DATA AGE
eating-bobcat-sonarqube-config 0 1s
eating-bobcat-sonarqube-copy-plugins 1 1s
eating-bobcat-sonarqube-install-plugins 1 1s
eating-bobcat-sonarqube-tests 1 1s
==> v1/PersistentVolumeClaim
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
eating-bobcat-postgresql Pending 1s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
eating-bobcat-postgresql-d6d94dcb4-cg2nq 0/1 Pending 0 0s
eating-bobcat-sonarqube-7c84f8b789-r6zm7 0/1 Init:0/1 0 0s
==> v1/Secret
NAME TYPE DATA AGE
eating-bobcat-postgresql Opaque 1 1s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
eating-bobcat-postgresql ClusterIP 10.43.140.71 <none> 5432/TCP 0s
eating-bobcat-sonarqube LoadBalancer 10.43.175.110 <pending> 9000:31481/TCP 0s
==> v1beta1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
eating-bobcat-postgresql 0/1 1 0 0s
eating-bobcat-sonarqube 0/1 1 0 0s
NOTES:
1. Get the application URL by running these commands:
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status of by running 'kubectl get svc -w eating-bobcat-sonarqube'
export SERVICE_IP=$(kubectl get svc --namespace default eating-bobcat-sonarqube -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo http://$SERVICE_IP:9000
Ahh.. but we forgot - k3s has no default storage class so we’ll have a pvc hung..
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
eating-bobcat-postgresql Pending 26s
Let’s set a storage class, make it default and try again:
JOHNSI10-M1:Documents johnsi10$ kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml
namespace/local-path-storage created
serviceaccount/local-path-provisioner-service-account created
clusterrole.rbac.authorization.k8s.io/local-path-provisioner-role created
clusterrolebinding.rbac.authorization.k8s.io/local-path-provisioner-bind created
deployment.apps/local-path-provisioner created
storageclass.storage.k8s.io/local-path created
configmap/local-path-config created
JOHNSI10-M1:Documents johnsi10$ kubectl get storageclass
NAME PROVISIONER AGE
local-path rancher.io/local-path 5s
JOHNSI10-M1:Documents johnsi10$ kubectl patch storageclass local-path -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'å
storageclass.storage.k8s.io/local-path patched
Try again…
JOHNSI10-M1:Documents johnsi10$ helm list
NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
eating-bobcat 1 Sat Oct 5 13:35:26 2019 DEPLOYED sonarqube-2.3.0 7.9 default
JOHNSI10-M1:Documents johnsi10$ helm delete eating-bobcat
release "eating-bobcat" deleted
JOHNSI10-M1:Documents johnsi10$ helm install --name mytest2 stable/sonarqube --set=postgresql.persistence.storageClass=local-path,persistence.storageClass=local-path
NAME: mytest2
LAST DEPLOYED: Sat Oct 5 14:11:56 2019
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/ConfigMap
NAME DATA AGE
mytest2-sonarqube-config 0 0s
mytest2-sonarqube-copy-plugins 1 0s
mytest2-sonarqube-install-plugins 1 0s
mytest2-sonarqube-tests 1 0s
==> v1/PersistentVolumeClaim
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
mytest2-postgresql Pending local-path 0s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
mytest2-postgresql-569679476d-qw8t4 0/1 Pending 0 0s
mytest2-sonarqube-7bfc4cf66f-ktlbs 0/1 Pending 0 0s
==> v1/Secret
NAME TYPE DATA AGE
mytest2-postgresql Opaque 1 0s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
mytest2-postgresql ClusterIP 10.43.156.202 <none> 5432/TCP 0s
mytest2-sonarqube LoadBalancer 10.43.160.188 <pending> 9000:30171/TCP 0s
==> v1beta1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
mytest2-postgresql 0/1 1 0 0s
mytest2-sonarqube 0/1 1 0 0s
NOTES:
1. Get the application URL by running these commands:
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status of by running 'kubectl get svc -w mytest2-sonarqube'
export SERVICE_IP=$(kubectl get svc --namespace default mytest2-sonarqube -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo http://$SERVICE_IP:9000
While the PVCs were satisfied, the main pod kept crashing:
Every 2.0s: kubectl get pods --all-namespaces JOHNSI10-M1: Sat Oct 5 15:16:36 2019
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-66f496764-s5bgd 1/1 Running 0 3h32m
kube-system helm-install-traefik-m6jlr 0/1 Completed 0 3h32m
kube-system svclb-traefik-5z86d 3/3 Running 0 3h32m
kube-system traefik-d869575c8-6pts7 1/1 Running 0 3h32m
kube-system tiller-deploy-8557598fbc-r7mgs 1/1 Running 0 3h6m
kube-system svclb-traefik-f9sq6 3/3 Running 0 3h9m
default svclb-mytest2-sonarqube-lgwml 1/1 Running 0 64m
default mytest2-postgresql-569679476d-qw8t4 1/1 Running 0 64m
local-path-storage local-path-provisioner-ccbdd96dc-xcsft 1/1 Running 0 21m
default svclb-mytest2-sonarqube-fk4lz 1/1 Running 0 64m
default mytest2-sonarqube-7bfc4cf66f-kxzcn 0/1 Running 5 17m
So let's try adding another worker
JOHNSI10-M1:Documents johnsi10$ multipass launch --name foo3
Launched: foo3
JOHNSI10-M1:Documents johnsi10$ multipass shell foo3
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-65-generic x86_64)
multipass@foo3:~$ sudo su -
root@foo3:~# curl -sfL https://get.k3s.io | K3S_URL=https://192.168.64.4:6443 K3S_TOKEN=K109b57bbfe0e3251481743ed00c79a323e9792eababc18ff7505a7e10f84c8e6cf::node:c9416ab5ca8157b919e331b1a4a0878d sh -
JOHNSI10-M1:Documents johnsi10$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
foo Ready worker 3h15m v1.15.4-k3s.1
foo2 Ready master 3h38m v1.15.4-k3s.1
foo3 Ready worker 12s v1.15.4-k3s.1
And we can manually try restarting the pod
default mytest2-sonarqube-7bfc4cf66f-kxzcn 0/1 Running 7 24m
JOHNSI10-M1:Documents johnsi10$ kubectl delete pod mytest2-sonarqube-7bfc4cf66f-kxzcn
pod "mytest2-sonarqube-7bfc4cf66f-kxzcn" deleted
I'll save you a lot more debugging and "watch" commands. In the end, I really just needed to create a worker with sufficient CPU and RAM. SQube is a rather fat java app and the meager memory requirements were causing the health checks to fail during startup.
Then and only then did the application pod finally come up:
JOHNSI10-M1:elasticsearch-cloud-deploy johnsi10$ multipass launch -c 2 -m 2G -n foo4
Launched: foo4
JOHNSI10-M1:elasticsearch-cloud-deploy johnsi10$ multipass shell foo4
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-65-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Sat Oct 5 15:41:42 CDT 2019
System load: 1.13 Processes: 125
Usage of /: 20.8% of 4.67GB Users logged in: 0
Memory usage: 6% IP address for enp0s2: 192.168.64.6
Swap usage: 0%
0 packages can be updated.
0 updates are security updates.
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
multipass@foo4:~$ sudo su -
root@foo4:~# curl -sfL https://get.k3s.io | K3S_URL=https://192.168.64.4:6443 K3S_TOKEN=K109b57bbfe0e3251481743ed00c79a323e9792eababc18ff7505a7e10f84c8e6cf::node:c9416ab5ca8157b919e331b1a4a0878d sh -
and one more restart
JOHNSI10-M1:Documents johnsi10$ kubectl delete pod mytest2-sonarqube-7bfc4cf66f-st7nm
pod "mytest2-sonarqube-7bfc4cf66f-st7nm" deleted
JOHNSI10-M1:Documents johnsi10$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
...
default mytest2-sonarqube-7bfc4cf66f-mffp9 1/1 Running 0 9m26s
JOHNSI10-M1:Documents johnsi10$ kubectl describe pod mytest2-sonarqube-7bfc4cf66f-mffp9
Name: mytest2-sonarqube-7bfc4cf66f-mffp9
Namespace: default
Priority: 0
PriorityClassName: <none>
Node: foo4/192.168.64.6
Start Time: Sat, 05 Oct 2019 15:42:21 -0500
Labels: app=sonarqube
pod-template-hash=7bfc4cf66f
release=mytest2
Annotations: <none>
Status: Running
IP: 10.42.3.4
K3s worked great, but could we use this for a full k8s cluster?
Kubernetes with Kubespray on Multipass
Following the kubespray guide from a previous blog entry we had (The Other Clouds: Vultr) let’s install k8s.
root@ks000b:/home/multipass/kubespray# cat inventory/mycluster/hosts.yaml
all:
hosts:
node1:
ansible_host: 192.168.64.7
ip: 192.168.64.7
access_ip: 192.168.64.7
node2:
ansible_host: 192.168.64.8
ip: 192.168.64.8
access_ip: 192.168.64.8
node3:
ansible_host: 192.168.64.9
ip: 192.168.64.9
access_ip: 192.168.64.9
children:
kube-master:
hosts:
node1:
kube-node:
hosts:
node1:
node2:
node3:
etcd:
hosts:
node1:
k8s-cluster:
children:
kube-master:
kube-node:
calico-rr:
hosts: {}We followed the linked guide and after a few fixing of root ssh keys and authorized hosts, as well as py2 and 3 installs, we wrapped up the k8s install
TASK [kubernetes/preinstall : run xfs_growfs] *****************************************************************************************************************
Saturday 05 October 2019 22:59:31 -0500 (0:00:00.199) 0:18:39.727 ******
PLAY RECAP ****************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
node1 : ok=709 changed=144 unreachable=0 failed=0 skipped=933 rescued=0 ignored=1
node2 : ok=447 changed=88 unreachable=0 failed=0 skipped=508 rescued=0 ignored=0
node3 : ok=447 changed=88 unreachable=0 failed=0 skipped=508 rescued=0 ignored=0
Saturday 05 October 2019 22:59:31 -0500 (0:00:00.185) 0:18:39.913 ******
===============================================================================
download : download_container | Download image if required ------------------------------------------------------------------------------------------- 107.62s
container-engine/docker : ensure docker packages are installed --------------------------------------------------------------------------------------- 102.11s
download : download_file | Download item -------------------------------------------------------------------------------------------------------------- 85.97s
download : download_file | Download item -------------------------------------------------------------------------------------------------------------- 46.43s
download : download_file | Download item -------------------------------------------------------------------------------------------------------------- 45.35s
bootstrap-os : Install python ------------------------------------------------------------------------------------------------------------------------- 45.33s
kubernetes/kubeadm : Join to cluster ------------------------------------------------------------------------------------------------------------------ 34.98s
kubernetes/master : kubeadm | Initialize first master ------------------------------------------------------------------------------------------------- 32.56s
download : download_container | Download image if required -------------------------------------------------------------------------------------------- 30.57s
download : download_file | Download item -------------------------------------------------------------------------------------------------------------- 25.77s
download : download_container | Download image if required -------------------------------------------------------------------------------------------- 21.53s
download : download_container | Download image if required -------------------------------------------------------------------------------------------- 20.51s
download : download_container | Download image if required -------------------------------------------------------------------------------------------- 19.50s
download : download_container | Download image if required -------------------------------------------------------------------------------------------- 15.30s
download : download_container | Download image if required -------------------------------------------------------------------------------------------- 14.83s
download : download_container | Download image if required -------------------------------------------------------------------------------------------- 11.86s
kubernetes/preinstall : Install packages requirements ------------------------------------------------------------------------------------------------- 10.87s
download : download_file | Download item -------------------------------------------------------------------------------------------------------------- 10.78s
kubernetes/master : Master | wait for kube-scheduler --------------------------------------------------------------------------------------------------- 9.82s
download : download_container | Download image if required --------------------------------------------------------------------------------------------- 7.49s
root@ks000b:/home/multipass/kubespray#
I then pulled the kubeconfig
root@node1:~# cat ~/.kube/config
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRFNU1UQXdOakF6TlRZek5Gb1hEVEk1TVRBd016QXpOVFl6TkZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTGIrCm1tNngxd0lxckdDOUczRTF2VlhBRmtZdExqeEQ5QmVUTURQVGNBYzVNaFBZY0Q3Qm5jbG00dy9hK1FweGpGWWoKZmRFaUJuQkZaRnFtZkY0ZGZIc3lSbzBFNzdLTmJjc2dCdTBldE13UG9JUGFZRDU5RUtDbjBDenl6THdSU1BaNwpsOGl2WE5iWEgrRndMcWdiY3NiaXBmNlFUN2RUUlRtM3RvVW53QklXRjVVNGRsNjVzMmtOUkQxWWVQRVliMWNnCjJnNW5sSUt0V3VjNzlWNDlZSjh5Qm1MNFV3NkhVdXZFeHl6MXpkWExyWWVrci9ybTdhVWw0U0RYWm96cEJSS3AKNFZKQ21nYkVZcWpoQXMybE5RTjB3aFk3VWx2NlVJNzRKR1JKVkdhejhrVzRGdm5QUThQWW1tRlB3VnhFUzhFMgpXbVZLNnNvVmtNWmRiUHFKcnlNQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFLU1ZnTmlQWkhURkEyaHRIQlV1cy9xVG93R1kKL3JscGdaR09DKy9yaUxCalBLOElWSWUydXVzUjFzOXZaWjVIeCtMTWNqVCtMUGdqcWJFdkI4dHFLTk4wZTdRRwp4VFNEenAxWmFnMmVKa1Y5WXQ1UnJNZVZCaG91Z3BPZm55NUR1QlRUaVFTeXVYN3pWWUIzZGsvTVMrdWFmVWttCng1elFCd0FGYVhsUFZ6RWpaL2V2T1N3U2JhS3Z6N3VWRlNTUGdlSmpiOXNoM01hRG1Keml5TW1IMVQvTUhpU0MKeXZvSzB5NDlnQjVEdS81Z1dNN0swRUV6U2RxbmtOSGhDOGJGL1lsSzM2VndyYU9WcW04YS9zODhOSTF4UE96NwpobDQ2eTM0YnVGMDQ4a3BNVHNLV1p3cWpOVWJJUlNXeDB3MURrenQreEp2OTNrRjJVanI5dHJvVk5oMD0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
server: https://192.168.64.7:6443
and checked for running pods
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-kube-controllers-586d5d67cb-gvvnd 1/1 Running 0 3m44s
kube-system calico-node-ndqsh 1/1 Running 0 4m28s
kube-system calico-node-whr5v 1/1 Running 1 4m28s
kube-system calico-node-zhl8g 1/1 Running 0 4m28s
kube-system coredns-58687784f9-7dw4l 1/1 Running 0 3m4s
kube-system coredns-58687784f9-jfvsp 1/1 Running 0 3m23s
kube-system dns-autoscaler-79599df498-76rd6 1/1 Running 0 3m19s
kube-system kube-apiserver-node1 1/1 Running 0 5m28s
kube-system kube-controller-manager-node1 1/1 Running 0 5m28s
kube-system kube-proxy-sjhvs 1/1 Running 0 4m42s
kube-system kube-proxy-tcdpj 1/1 Running 0 4m42s
kube-system kube-proxy-tmdvf 1/1 Running 0 4m42s
kube-system kube-scheduler-node1 1/1 Running 0 5m28s
kube-system kubernetes-dashboard-556b9ff8f8-nxvpf 1/1 Running 0 3m15s
kube-system nginx-proxy-node2 1/1 Running 0 4m44s
kube-system nginx-proxy-node3 1/1 Running 0 4m46s
kube-system nodelocaldns-pj29s 1/1 Running 0 3m18s
kube-system nodelocaldns-qfgdm 1/1 Running 0 3m18s
kube-system nodelocaldns-qjjrq 1/1 Running 0 3m18s
We can port-forward to the dashboard
JOHNSI10-M1:Documents johnsi10$ kubectl port-forward kubernetes-dashboard-556b9ff8f8-nxvpf -n kube-system 8443:8443
Forwarding from 127.0.0.1:8443 -> 8443
Forwarding from [::1]:8443 -> 8443
Handling connection for 8443
One note, getting a functional service user and token took a few. In short, you’ll want to apply a ClusterAdmin role to the default token
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kube-system
- kind: ServiceAccount
name: default
namespace: default
Summary
Ubuntu multipass offers a nice fast alternative to Docker. It uses HyperKit for MacOS under the covers (more here https://github.com/CanonicalLtd/multipass) and was rather easy to get a k3s and k8s cluster up and running.
It supports metadata for cloud-init - basically you can create a config file to set up users and groups and even run arbitrary setup code. This means one could likely automate the creation of local clusters.
I hope to dig into multipass on Windows next time and explore cloud-init in greater detail.