Published: Mar 9, 2023 by Isaac Johnson
I have come across NGrok in a few tutorials over time. It provides a fantastic service that is basically a dynamic DNS plus reverse proxy in one. It makes exposing ones internal service fantastically easy and the basic plan is free.
Signup
Let’s create a free plan from the main NGrok page
I chose to federate through google. We arrive at the main dashboard
Let’s start with a simple test. Whenever I write these, I fire up bundle exec jekyll serve and use https://localhost:4000 to check the site.
So now I’ll pull down the ngrok binary into WSL
builder@DESKTOP-72D2D9T:~$ wget https://bin.equinox.io/c/bNyj1mQVY4c/ngrok-v3-stable-linux-amd64.tgz
--2023-03-06 18:52:55-- https://bin.equinox.io/c/bNyj1mQVY4c/ngrok-v3-stable-linux-amd64.tgz
Resolving bin.equinox.io (bin.equinox.io)... 52.202.168.65, 54.237.133.81, 18.205.222.128, ...
Connecting to bin.equinox.io (bin.equinox.io)|52.202.168.65|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9300212 (8.9M) [application/octet-stream]
Saving to: ‘ngrok-v3-stable-linux-amd64.tgz’
ngrok-v3-stable-linux-amd64.t 100%[=================================================>] 8.87M 2.41MB/s in 4.8s
2023-03-06 18:53:00 (1.86 MB/s) - ‘ngrok-v3-stable-linux-amd64.tgz’ saved [9300212/9300212]
builder@DESKTOP-72D2D9T:~$ tar -xzvf ngrok-v3-stable-linux-amd64.tgz
ngrok
I’ll then configure it
builder@DESKTOP-72D2D9T:~$ chmod 755 ./ngrok
builder@DESKTOP-72D2D9T:~$ ./ngrok config add-authtoken asdfjhiasdfjhkjfsdjksdfakjlsdfasdafjsdafdsfaasd
Authtoken saved to configuration file: /home/builder/.config/ngrok/ngrok.yml
I’ll now fire up NGrok in a seperate window
which I can now use to visit the site
This means the site, as running on my laptop is publicly available on that URL as long as ngrok is running. The free plan means it won’t keep that URL if I restart.
I can see the traffic served when looking at the ngrok window
SSH tunnels
Say I wanted to, for a short period, expose my WSL to an outside source.
I could fire up a 22 forwarder:
$ ./ngrok tcp 22
Here I’ll snap install Ngrok to a Linux host in my network
isaac@isaac-MacBookAir:~$ sudo snap install ngrok
ngrok (v3/stable) 3.1.0 from Kyle Wenholz (kyle-ngrok) installed
isaac@isaac-MacBookAir:~$ ngrok config add-authtoken asdfjhiasdfjhkjfsdjksdfakjlsdfasdafjsdafdsfaasd
Authtoken saved to configuration file: /home/isaac/snap/ngrok/89/.config/ngrok/ngrok.yml
isaac@isaac-MacBookAir:~$ ngrok tcp 22
Kubernetes
For this test, I’ll use a pretty basic 3 node cluster that has Traefik for Ingress already
builder@DESKTOP-QADGF36:~/Workspaces/ansible-playbooks$ kubectl get ingress --all-namespaces
No resources found
builder@DESKTOP-QADGF36:~/Workspaces/ansible-playbooks$ kubectl get svc --all-namespaces
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 11h
kube-system kube-dns ClusterIP 10.43.0.10 <none> 53/UDP,53/TCP,9153/TCP 11h
kube-system metrics-server ClusterIP 10.43.65.92 <none> 443/TCP 11h
kube-system traefik LoadBalancer 10.43.239.133 192.168.1.12,192.168.1.159,192.168.1.206 80:30246/TCP,443:31771/TCP 11h
So we have something to look at, I’ll fire up the Azure Vote Sample App
$ helm repo add azure-samples https://azure-samples.github.io/helm-charts/
"azure-samples" already exists with the same configuration, skipping
builder@DESKTOP-QADGF36:~/Workspaces/ansible-playbooks$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "longhorn" chart repository
...Successfully got an update from the "kuma" chart repository
...Successfully got an update from the "freshbrewed" chart repository
...Successfully got an update from the "myharbor" chart repository
...Successfully got an update from the "dapr" chart repository
...Successfully got an update from the "confluentinc" chart repository
...Successfully got an update from the "sonarqube" chart repository
...Successfully got an update from the "azure-samples" chart repository
...Successfully got an update from the "kubecost" chart repository
...Successfully got an update from the "actions-runner-controller" chart repository
...Successfully got an update from the "hashicorp" chart repository
...Successfully got an update from the "sumologic" chart repository
...Successfully got an update from the "novum-rgi-helm" chart repository
...Successfully got an update from the "adwerx" chart repository
...Successfully got an update from the "open-telemetry" chart repository
...Successfully got an update from the "rhcharts" chart repository
...Successfully got an update from the "epsagon" chart repository
...Successfully got an update from the "lifen-charts" chart repository
...Successfully got an update from the "uptime-kuma" chart repository
...Successfully got an update from the "elastic" chart repository
...Successfully got an update from the "datadog" chart repository
...Successfully got an update from the "nginx-stable" chart repository
...Successfully got an update from the "rook-release" chart repository
...Successfully got an update from the "harbor" chart repository
...Successfully got an update from the "argo-cd" chart repository
...Successfully got an update from the "incubator" chart repository
...Successfully got an update from the "newrelic" chart repository
...Successfully got an update from the "crossplane-stable" chart repository
...Successfully got an update from the "gitlab" chart repository
...Successfully got an update from the "rancher-latest" chart repository
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈
$ helm install azure-samples/azure-vote --generate-name
NAME: azure-vote-1678237213
LAST DEPLOYED: Tue Mar 7 19:00:14 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The Azure Vote application has been started on your Kubernetes cluster.
Title: Azure Vote App
Vote 1 value: Cats
Vote 2 value: Dogs
The externally accessible IP address can take a minute or so to provision. Run the following command to monitor the provisioning status. Once an External IP address has been provisioned, brows to this IP address to access the Azure Vote application.
kubectl get service -l name=azure-vote-front -w
We’ll see a service was created, but I do not expect an external-IP to be provided
$ kubectl get service -l name=azure-vote-front -w
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
azure-vote-front LoadBalancer 10.43.245.23 <pending> 80:31757/TCP 67s
and the pods are running
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
vote-back-azure-vote-1678237213-5b7d969f66-zcx85 1/1 Running 0 87s
vote-front-azure-vote-1678237213-79974bfc78-ss8ps 1/1 Running 0 87s
If I port-forward, I can see the App
A simple NGrok Deployment forwarder
This method was first laid out in this medium post several years ago. Though I needed to tweak to add the authtoken which is required now.
The Deployment YAML:
$ cat testNgrok.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: ngrok
spec:
selector:
matchLabels:
app: ngrok
replicas: 1
template:
metadata:
labels:
app: ngrok
spec:
containers:
- name: ngrok
image: wernight/ngrok
command: ["ngrok"]
args: ["http", "-authtoken=asdfjhiasdfjhkjfsdjksdfakjlsdfasdafjsdafdsfaasd", "azure-vote-front"]
ports:
- containerPort: 4040
$ kubectl apply -f ./testNgrok.yml
deployment.apps/ngrok created
Once launched, I’ll query the port 4040 Tunnel API Endpoint to see the NGrok URL
$ kubectl exec `kubectl get pods -l=app=ngrok -o=jsonpath='{.items[0].metadata.name}'` -- curl http://localhost:4040/api/tunnels | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 799 100 799 0 0 588k 0 --:--:-- --:--:-- --:--:-- 780k
{
"tunnels": [
{
"name": "command_line",
"uri": "/api/tunnels/command_line",
"public_url": "https://09cf-73-242-50-46.ngrok.io",
"proto": "https",
"config": {
"addr": "http://azure-vote-front:80",
"inspect": true
},
"metrics": {
"conns": {
"count": 0,
"gauge": 0,
"rate1": 0,
"rate5": 0,
"rate15": 0,
"p50": 0,
"p90": 0,
"p95": 0,
"p99": 0
},
"http": {
"count": 0,
"rate1": 0,
"rate5": 0,
"rate15": 0,
"p50": 0,
"p90": 0,
"p95": 0,
"p99": 0
}
}
},
{
"name": "command_line (http)",
"uri": "/api/tunnels/command_line%20%28http%29",
"public_url": "http://09cf-73-242-50-46.ngrok.io",
"proto": "http",
"config": {
"addr": "http://azure-vote-front:80",
"inspect": true
},
"metrics": {
"conns": {
"count": 0,
"gauge": 0,
"rate1": 0,
"rate5": 0,
"rate15": 0,
"p50": 0,
"p90": 0,
"p95": 0,
"p99": 0
},
"http": {
"count": 0,
"rate1": 0,
"rate5": 0,
"rate15": 0,
"p50": 0,
"p90": 0,
"p95": 0,
"p99": 0
}
}
}
],
"uri": "/api/tunnels"
}
You can see, we have exposed both HTTP and HTTPS
And it’s just as easy to remove
$ kubectl delete -f testNgrok.yml
deployment.apps "ngrok" deleted
As an aside, if you don’t have a valid Auth key or it’s missing, expect an error
Unofficial Official Helm Chart
Note: I’m not successful but that’s because this path only works for paid accounts
We already have our Auth Token as we used it to configure our other instances
But we need to get an API key
I’ll click create an optionally change its name
When we click create, we’ll be shown the key once
We can see the Unofficial NGrok Ingress Controller Helm Chart for a more scalable way to use NGrok.
First we add the Helm chart
$ helm repo add ngrok https://ngrok.github.io/kubernetes-ingress-controller
"ngrok" has been added to your repositories
$ helm repo update
Then we can install
$ helm install ngrok-ingress-controller ngrok/kubernetes-ingress-controller --namespace ngrok-ingress-controller --create-namespace --set credentials.apiKey=ruruywreuiweuitruiwiureutrerreterter --set credentials.authtoken=asdfjhiasdfjhkjfsdjksdfakjlsdfasdafjsdafdsfaasd --set ingressClass.default=true
NAME: ngrok-ingress-controller
LAST DEPLOYED: Tue Mar 7 19:37:04 2023
NAMESPACE: ngrok-ingress-controller
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
================================================================================
The ngrok Ingress controller has been deployed as a Deployment type to your
cluster.
If you haven't yet, create some Ingress resources in your cluster and they will
be automatically configured on the internet using ngrok.
One example, taken from your cluster, is the Service:
"metrics-server"
You can make this accessible via Ngrok with the following manifest:
--------------------------------------------------------------------------------
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: metrics-server
namespace: kube-system
spec:
ingressClassName: ngrok
rules:
- host: metrics-server-3mdBbZaf.ngrok.io
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: metrics-server
port:
number: 443
--------------------------------------------------------------------------------
Applying this manifest will make the service "metrics-server"
available on the public internet at "https://metrics-server-3mdBbZaf.ngrok.io/".
Once done, view your edges in the Dashboard https://dashboard.ngrok.com/cloud-edge/edges
Find the tunnels running in your cluster here https://dashboard.ngrok.com/tunnels/agents
This controller is under active development and is in Alpha. It is not suggested to use this in production.
If you have any questions or feedback, please join us in https://ngrok.com/slack and let us know!
Sadly, I cannot just update the Azure Vote App helm values to use the new Ingress type as that isn’t exposed
$ helm get values --all azure-vote-1678237213
COMPUTED VALUES:
serviceNameFront: azure-vote-front
serviceType: LoadBalancer
title: Azure Vote App
value1: Cats
value2: Dogs
I’ll delete the old Azure Vote helm then redo to use the new default Ingress controller
$ helm delete azure-vote-1678237213
release "azure-vote-1678237213" uninstalled
$ helm install azure-samples/azure-vote --generate-name
NAME: azure-vote-1678239487
LAST DEPLOYED: Tue Mar 7 19:38:08 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The Azure Vote application has been started on your Kubernetes cluster.
Title: Azure Vote App
Vote 1 value: Cats
Vote 2 value: Dogs
The externally accessible IP address can take a minute or so to provision. Run the following command to monitor the provisioning status. Once an External IP address has been provisioned, brows to this IP address to access the Azure Vote application.
kubectl get service -l name=azure-vote-front -w
However, this does work
builder@DESKTOP-QADGF36:~/Workspaces/ansible-playbooks$ cat testAVFIngress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: azure-vote
spec:
ingressClassName: ngrok
rules:
- host: azure-vote-3mdbbzaf.ngrok.io
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: azure-vote-front
port:
number: 80
builder@DESKTOP-QADGF36:~/Workspaces/ansible-playbooks$ kubectl apply -f testAVFIngress.yaml
ingress.networking.k8s.io/azure-vote created
builder@DESKTOP-QADGF36:~/Workspaces/ansible-playbooks$ kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
azure-vote ngrok azure-vote-3mdbbzaf.ngrok.io 80 10s
But it does not work - we can see the normal ngrok page when it is not a valid URL https://azure-vote-3mdbbzaf.ngrok.io/
Looking at the logs, I see plenty of errors
2023-03-08T01:37:26Z INFO cache-store-driver syncing driver state!!
2023-03-08T01:37:26Z DEBUG events Deleted Domain azure-vote-gdhwbgtg-ngrok-io {"type": "Normal", "object": {"kind":"Domain","namespace":"default","name":"azure-vote-gdhwbgtg-ngrok-io
","uid":"d37da700-dd7c-4593-93ac-4a38d971ee50","apiVersion":"ingress.k8s.ngrok.com/v1alpha1","resourceVersion":"18867"}, "reason": "Deleted"}
2023-03-08T01:37:26Z DEBUG events Deleted Edge azure-vote-gdhwbgtg-ngrok-io {"type": "Normal", "object": {"kind":"HTTPSEdge","namespace":"default","name":"azure-vote-gdhwbgtg-ngrok
-io","uid":"afd39ff7-e1ca-47de-8832-3f6332424a3e","apiVersion":"ingress.k8s.ngrok.com/v1alpha1","resourceVersion":"18868"}, "reason": "Deleted"}
2023-03-08T01:37:26Z ERROR controllers.https-edge unable to fetch Edge {"V1Alpha1HTTPSEdge": "default/azure-vote-gdhwbgtg-ngrok-io", "error": "HTTPSEdge.ingress.k8s.ngrok.com \"azure-
vote-gdhwbgtg-ngrok-io\" not found"}
E0308 01:37:32.404626 1 reflector.go:140] pkg/mod/k8s.io/client-go@v0.26.0/tools/cache/reflector.go:169: Failed to watch *v1alpha1.Domain: the server could not find the requested resourc
e (get domains.ingress.k8s.ngrok.com)
E0308 01:37:32.434640 1 reflector.go:140] pkg/mod/k8s.io/client-go@v0.26.0/tools/cache/reflector.go:169: Failed to watch *v1alpha1.HTTPSEdge: the server could not find the requested reso
urce (get httpsedges.ingress.k8s.ngrok.com)
W0308 01:37:33.491187 1 reflector.go:424] pkg/mod/k8s.io/client-go@v0.26.0/tools/cache/reflector.go:169: failed to list *v1alpha1.Domain: the server could not find the requested resource
(get domains.ingress.k8s.ngrok.com)
E0308 01:37:33.491323 1 reflector.go:140] pkg/mod/k8s.io/client-go@v0.26.0/tools/cache/reflector.go:169: Failed to watch *v1alpha1.Domain: failed to list *v1alpha1.Domain: the server cou
ld not find the requested resource (get domains.ingress.k8s.ngrok.com)
The repeated error is
2023-03-08T12:02:35Z INFO cache-store-driver syncing driver state!!
2023-03-08T12:02:35Z ERROR cache-store-driver error creating domain {"domain": {"metadata":{"name":"azure-vote-3mdbbzaf-ngrok-io","namespace":"default","creationTimestamp":null},"spec":{"domain":"azure-vote-3mdbbzaf.ngrok.io"},"status":{}}, "error": "the server could not find the requested resource (post domains.ingress.k8s.ngrok.com)"}
2023-03-08T12:02:45Z ERROR controllers.ingress Failed to sync ingress to store {"error": "the server could not find the requested resource (post domains.ingress.k8s.ngrok.com)"}
2023-03-08T12:02:45Z ERROR Reconciler error {"controller": "ingress", "controllerGroup": "networking.k8s.io", "controllerKind": "Ingress", "Ingress": {"name":"azure-vote","namespace":"default"}, "namespace": "default", "name": "azure-vote", "reconcileID": "d7628c86-6e04-42af-8305-8521e7076550", "error": "the server could not find the requested resource (post domains.ingress.k8s.ngrok.com)"}
Perhaps this requires a paid account.
Redoing it all I found that the error now is rather clear
2023-03-08T12:29:57Z DEBUG events Failed to create Domain azure-vote-3mdbbzaf-ngrok-io: HTTP 400: Only Personal, Pro or Enterprise accounts can reserve domains.
Your account can't reserve domains.
Upgrade to a paid plan at: https://dashboard.ngrok.com/billing/subscription [ERR_NGROK_401]
Operation ID: op_2MjQdOarUYbzOntLfQpoc1Zb3zM {"type": "Warning", "object": {"kind":"Domain","namespace":"default","name":"azure-vote-3mdbbzaf-ngrok-io","uid":"cf399f04-fd8a-49e2-ae0d-6356fe4d2127","apiVersion":"ingress.k8s.ngrok.com/v1alpha1","resourceVersion":"1108"}, "reason": "CreateFailed"}
2023-03-08T12:29:58Z ERROR Reconciler error {"controller": "httpsedge", "controllerGroup": "ingress.k8s.ngrok.com", "controllerKind": "HTTPSEdge", "HTTPSEdge": {"name":"azure-vote-3mdbbzaf-ngrok-io","namespace":"default"}, "namespace": "default", "name": "azure-vote-3mdbbzaf-ngrok-io", "reconcileID": "c61ac568-af7c-4d66-834c-3eab686e9e3b", "error": "HTTP 400: The domain 'azure-vote-3mdbbzaf.ngrok.io:443' is not reserved. [ERR_NGROK_7117]\n\nOperation ID: op_2MjQdOXEGjpzMcSq3SFr8s83VDU"}
which lines up to the lack of domain reservation
Costs
By month, NGrok ranges from $10 to $75. You can save 20% billing annually.
All the accounts are limited to 1Gb a month. Most basic websites can handle that. But when you get into hosting artifact stores, containers, or perhaps video you can blow past that with ease.
In the “Pro” and “Enterprise” tiers you can optionally enable On-Demand pricing which would enable additional bandwidth at $0.10/Gb
I know for my home internet I routinely touch upward of a Tb - as I have a residential provider with Data Caps I have to pay extra to remove (grumble grumble) and this site generates upwards of 38Gb of traffic from a static site
At 37.21Gb in a month, that would be about $4. To handle all the traffic from my house would be $100/mo. That isn’t counting the other overages such as Peak usage or additional agents.
Other features
Most other features are behind the paywall
TLS Certs:
IP Policies:
SSO:
Domains:
Summmary
Ngrok solves a problem. It creates a basic Dynamic DNS reverse tunnel easily. It works with most OSes and certainly can work in Kubernetes. The free offering is great for demos or quick checks. I actually like that they have some lower cost paid tiers.
Think of the following scenarios:
- you have a legacy system on prem that cannot move to the cloud - due to age, rules, software - but you need to expose this middleware to an application
- you have a restricted device like a nanycam or a NAS with photo software - you want to share with family or check when you are remote - low bandwidth
- you have a basic webhook, function or tool on-prem
- you have a legacy revision control system that by rules you cannot expose but it’s auth is insufficient (think SourceSafe or an old CVS host).
For some reason $10/mo is on my line for random stuff I’ll pay for - I probably would have pulled the trigger at $5 month for doing demos but not $10.
