We started out last week with Dapr and followed the getting started tutorials. At that time we experimented with local containerized installs of Dapr as well as Kubernetes through AKS. This week we want to circle back on that pub-sub model and tackle two objectives: First, figure out how to test locally without needing a full running Dapr instance (work out the syntax of cloudevents) and secondly, mostly for fun, build a full subscriber microservice using Perl. Because why not. We will even extend the functionality to push messages to Teams channels.
Setup
Let’s get our DapR instance up and running
builder@DESKTOP-72D2D9T:~/Workspaces/dapr$ dapr init
⌛ Making the jump to hyperspace...
↓ Downloading binaries and setting up components...
Dapr runtime installed to /home/builder/.dapr/bin, you may run the following to add it to your path if you want to run daprd directly:
export PATH=$PATH:/home/builder/.dapr/bin
✅ Downloaded binaries and completed components set up.
ℹ️ daprd binary has been installed to /home/builder/.dapr/bin.
ℹ️ dapr_placement container is running.
ℹ️ dapr_redis container is running.
ℹ️ dapr_zipkin container is running.
ℹ️ Use `docker ps` to check running containers.
✅ Success! Dapr is up and running. To get started, go here: https://aka.ms/dapr-getting-started
Verification
builder@DESKTOP-72D2D9T:~/Workspaces/dapr$ dapr --version
CLI version: 1.0.1
Runtime version: 1.0.1
builder@DESKTOP-72D2D9T:~/Workspaces/dapr$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d166f4124555 daprio/dapr "./placement" 9 minutes ago Up 9 minutes 0.0.0.0:50005->50005/tcp dapr_placement
2c74b10b620a redis "docker-entrypoint.s…" 9 minutes ago Up 9 minutes 0.0.0.0:6379->6379/tcp dapr_redis
4038a9b6665e openzipkin/zipkin "start-zipkin" 9 minutes ago Up 9 minutes (healthy) 9410/tcp, 0.0.0.0:9411->9411/tcp dapr_zipkin
Perl
We need a few CPAN modules first.
builder@DESKTOP-72D2D9T:~/Workspaces/dapr/quickstarts/pub-sub$ sudo cpan install HTTP::Server::Simple JSON
[sudo] password for builder:
Loading internal logger. Log::Log4perl recommended for better logging
CPAN.pm requires configuration, but most of it can be done automatically.
If you answer 'no' below, you will enter an interactive dialog for each
configuration option instead.
Would you like to configure as much as possible automatically? [yes]
Fetching with HTTP::Tiny:
http://www.cpan.org/authors/01mailrc.txt.gz
Reading '/root/.cpan/sources/authors/01mailrc.txt.gz'
While we can publish with Dapr to containerized services in Dapr with dapr publish --publish-app-id react-form --pubsub pubsub --topic A --data '{ "message": "This is a test" }' we can also impersonate that call with curl as well.
A basic query might be:
curl -X POST http://localhost:8080/A -d "whatever&asdf=asdf"
or even against the node subscriber with
curl -X POST http://localhost:3000/A -d '{"data" : "{\"message\": \"howdy\", \"topic\",\"A\"}","topic" : "A"}' -H 'Content-Type: application/json'clearly we may want to see the subscribe output as well
curl -O http://localhost:8080/dapr/subscribeTo start with, here is my server.pl :
#!/usr/bin/perl
{
package MyWebServer;
use HTTP::Server::Simple::CGI;
use base qw(HTTP::Server::Simple::CGI);
my %dispatch = (
'/dapr/subscribe' => \&resp_subscribe,
'/hello' => \&resp_hello,
'/A' => \&resp_A,
# ...
);
sub handle_request {
my $self = shift;
my $cgi = shift;
my $path = $cgi->path_info();
my $handler = $dispatch{$path};
if (ref($handler) eq "CODE") {
print "HTTP/1.0 200 OK\r\n";
$handler->($cgi);
} else {
print "HTTP/1.0 404 Not found\r\n";
print $cgi->header,
$cgi->start_html('Not found'),
$cgi->h1('Not found'),
$cgi->end_html;
}
}
sub resp_hello {
my $cgi = shift; # CGI.pm object
return if !ref $cgi;
my $who = $cgi->param('name');
print $cgi->header,
$cgi->start_html("Hello"),
$cgi->h1("Hello $who!"),
$cgi->end_html;
}
sub resp_A {
my $cgi = shift; # CGI.pm object
return if !ref $cgi;
print $cgi->header('application/json');
print "\n";
print STDERR "\n";
print $cgi->param( 'POSTDATA' );
print STDERR $cgi->param( 'POSTDATA' );
}
sub resp_subscribe {
my $cgi = shift; # CGI.pm object
print $cgi->header('application/json');
my @rec_hash = ( { 'pubsubname' => 'pubsub', 'topic' => 'A', 'route' => 'A' }, { 'pubsubname' => 'pubsub', 'topic' => 'B', 'route' => 'B'}, { 'pubsubname' => 'pubsub', 'topic' => 'C', 'route' => 'C'} );
my $json = encode_json \@rec_hash;
#my %rec_hash = ('a' => 1, 'b' => 2, 'c' => 3, 'd' => 4, 'e' => 5);
#my $json = encode_json \%rec_hash;
print "$json\n";
}
}
# start the server on port 8080
my $pid = MyWebServer->new(8080)->background();
print "Use 'kill $pid' to stop server.\n";
However, I rather grew tired of re-initializing Dapr with computer reboots. It works locally, but my Windows 10 likes to update plenty (cost of fast ring) and it seems that i was often re-initializing the whole stack between updates. I also wasn’t keen on paying for an AKS stack just for Dapr.
This led me want to try setting it up on my home K3S cluster.
Setting up Dapr in K3s (onprem)
verify Dapr is installed
$ dapr --version
CLI version: 1.0.1
Runtime version: 1.0.1
Then init, this time with "-k". Note, i have the kubeconfig set for my k3s cluster locally:
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
isaac-macbookair Ready master 98d v1.19.5+k3s2
isaac-macbookpro Ready <none> 97d v1.19.5+k3s2$ dapr init -k -n default
⌛ Making the jump to hyperspace...
ℹ️ Note: To install Dapr using Helm, see here: https://docs.dapr.io/getting-started/install-dapr-kubernetes/#install-with-helm-advanced
✅ Deploying the Dapr control plane to your cluster...
✅ Success! Dapr has been installed to namespace default. To verify, run `dapr status -k' in your terminal. To get started, go here: https://aka.ms/dapr-getting-started
Verification
Checking status
$ dapr status -k
NAME NAMESPACE HEALTHY STATUS REPLICAS VERSION AGE CREATED
dapr-sidecar-injector default False Waiting (ContainerCreating) 1 1.1.0 7s 2021-04-02 08:37.23
dapr-placement-server default False Waiting (ContainerCreating) 1 1.1.0 7s 2021-04-02 08:37.23
dapr-sentry default False Waiting (ContainerCreating) 1 1.1.0 7s 2021-04-02 08:37.23
dapr-operator default False Waiting (ContainerCreating) 1 1.1.0 7s 2021-04-02 08:37.23
dapr-dashboard default False Waiting (ContainerCreating) 1 0.6.0 7s 2021-04-02 08:37.23
$ dapr status -k
NAME NAMESPACE HEALTHY STATUS REPLICAS VERSION AGE CREATED
dapr-sentry default True Running 1 1.1.0 1m 2021-04-02 08:37.23
dapr-sidecar-injector default True Running 1 1.1.0 1m 2021-04-02 08:37.23
dapr-placement-server default True Running 1 1.1.0 1m 2021-04-02 08:37.23
dapr-operator default True Running 1 1.1.0 1m 2021-04-02 08:37.23
dapr-dashboard default True Running 1 0.6.0 1m 2021-04-02 08:37.23
Then we need to add redis…
$ helm repo add bitnami https://charts.bitnami.com/bitnami
"bitnami" already exists with the same configuration, skipping
$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "dapr" chart repository
...Successfully got an update from the "hashicorp" chart repository
...Successfully got an update from the "kedacore" chart repository
...Successfully got an update from the "azure-samples" chart repository
...Successfully got an update from the "datadog" chart repository
...Successfully got an update from the "nginx-stable" chart repository
...Successfully got an update from the "datawire" chart repository
...Successfully got an update from the "openfaas" chart repository
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "bitnami" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈Happy Helming!⎈
$ helm install redis bitnami/redis
NAME: redis
LAST DEPLOYED: Fri Apr 2 08:40:08 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
** Please be patient while the chart is being deployed **
Redis(TM) can be accessed via port 6379 on the following DNS names from within your cluster:
redis-master.default.svc.cluster.local for read/write operations
redis-slave.default.svc.cluster.local for read-only operations
To get your password run:
export REDIS_PASSWORD=$(kubectl get secret --namespace default redis -o jsonpath="{.data.redis-password}" | base64 --decode)
To connect to your Redis(TM) server:
1. Run a Redis(TM) pod that you can use as a client:
kubectl run --namespace default redis-client --rm --tty -i --restart='Never' \
--env REDIS_PASSWORD=$REDIS_PASSWORD \
--image docker.io/bitnami/redis:6.0.12-debian-10-r3 -- bash
2. Connect using the Redis(TM) CLI:
redis-cli -h redis-master -a $REDIS_PASSWORD
redis-cli -h redis-slave -a $REDIS_PASSWORD
To connect to your database from outside the cluster execute the following commands:
kubectl port-forward --namespace default svc/redis-master 6379:6379 &
redis-cli -h 127.0.0.1 -p 6379 -a $REDIS_PASSWORD
We can now get our redis password we’ll need later
$ kubectl get secret --namespace default redis -o jsonpath="{.data.redis-password}" | base64 --decode
vg3ZXFuMuo
And just to double check the name (should be redis-master):
$ kubectl get svc | grep redis
redis-headless ClusterIP None <none> 6379/TCP 111s
redis-slave ClusterIP 10.43.88.18 <none> 6379/TCP 111s
redis-master ClusterIP 10.43.237.246 <none> 6379/TCP 111s
Now we can apply it (in deploy/redis.yaml). A reminder we are using the "pub-sub" starts in the dapr/quickstarts repo.
builder@DESKTOP-JBA79RT:~/Workspaces/dapr$ cat pub-sub/deploy/redis.yaml
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: pubsub
spec:
type: pubsub.redis
version: v1
metadata:
- name: "redisHost"
value: "redis-master:6379"
- name: "redisPassword"
value: "vg3ZXFuMuo"
builder@DESKTOP-JBA79RT:~/Workspaces/dapr$ kubectl apply -f pub-sub/deploy/redis.yaml
component.dapr.io/pubsub created
Now just cd into the pub-sub/deploy folder to apply them all
$ kubectl apply -f .
deployment.apps/node-subscriber created
deployment.apps/python-subscriber created
service/react-form created
deployment.apps/react-form created
component.dapr.io/pubsub configured
While we cannot publish to the topic locally since publish doesnt support -k
$ dapr publish -k --publish-app-id react-form --pubsub pubsub --topic A --data '{ "message": "This is a test" }'
Error: unknown shorthand flag: 'k' in -k
Usage:
dapr publish [flags]
We can just as easily use the react form
$ kubectl get pods | grep react
react-form-7975b5fff9-db8ww 2/2 Running 1 13m
$ kubectl port-forward react-form-7975b5fff9-db8ww 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080
We can now see it from the node-app:
$ kubectl logs node-subscriber-7dc79579bc-v2s6m node-subscriber
Node App listening on port 3000!
A: This is a test
And the python app as well:
$ kubectl logs python-subscriber-6fd5dc7f8c-46v2t python-subscriber
* Serving Flask app "app" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [02/Apr/2021 13:53:44] "GET /dapr/config HTTP/1.1" 404 -
127.0.0.1 - - [02/Apr/2021 13:53:44] "GET /dapr/subscribe HTTP/1.1" 200 -
A: {'type': 'com.dapr.event.sent', 'pubsubname': 'pubsub', 'traceid': '00-8f5222adfb59d90e26ff4998026ce0ad-fa37e3f7d4eca539-00', 'data': {'messageType': 'A', 'message': 'This is a test'}, 'datacontenttype': 'application/json', 'specversion': '1.0', 'source': 'react-form', 'topic': 'A', 'id': '223efa8a-9228-491a-a4e2-d081a81e0c8a'}
Received message "This is a test" on topic "A"
127.0.0.1 - - [02/Apr/2021 14:08:10] "POST /A HTTP/1.1" 200 -
Perl app
Now lets go back to that server.pl
#!/usr/bin/perl
{
package MyWebServer;
use HTTP::Server::Simple::CGI;
use base qw(HTTP::Server::Simple::CGI);
use JSON;
my %dispatch = (
'/dapr/subscribe' => \&resp_subscribe,
'/hello' => \&resp_hello,
'/A' => \&resp_A,
'/B' => \&resp_B,
# ...
);
sub handle_request {
my $self = shift;
my $cgi = shift;
my $path = $cgi->path_info();
my $handler = $dispatch{$path};
if (ref($handler) eq "CODE") {
print "HTTP/1.0 200 OK\r\n";
$handler->($cgi);
} else {
print "HTTP/1.0 404 Not found\r\n";
print $cgi->header,
$cgi->start_html('Not found'),
$cgi->h1('Not found'),
$cgi->end_html;
}
}
sub resp_hello {
my $cgi = shift; # CGI.pm object
return if !ref $cgi;
my $who = $cgi->param('name');
print $cgi->header,
$cgi->start_html("Hello"),
$cgi->h1("Hello $who!"),
$cgi->end_html;
}
sub resp_A {
my $cgi = shift; # CGI.pm object
return if !ref $cgi;
print $cgi->header('application/json');
print $cgi->param( 'POSTDATA' );
#if ('POST' eq $c->request_method && $c->param('dl')) {
}
sub resp_B {
my $cgi = shift; # CGI.pm object
return if !ref $cgi;
print $cgi->header('application/json');
#print $cgi->param( 'POSTDATA' );
print $cgi->Dump;
#if ('POST' eq $c->request_method && $c->param('dl')) {
}
sub resp_subscribe {
my $cgi = shift; # CGI.pm object
print $cgi->header('application/json');
my @rec_hash = ( { 'pubsubname' => 'pubsub', 'topic' => 'A', 'route' => 'A' }, { 'pubsubname' => 'pubsub', 'topic' => 'B', 'route' => 'B'} );
my $json = encode_json \@rec_hash;
#my %rec_hash = ('a' => 1, 'b' => 2, 'c' => 3, 'd' => 4, 'e' => 5);
#my $json = encode_json \%rec_hash;
print "$json\n";
}
}
# start the server on port 8080
my $pid = MyWebServer->new(8080)->background();
print "Use 'kill $pid' to stop server.\n";
I've added some JSON parsing (though not complete) that we can test
builder@DESKTOP-JBA79RT:~/Workspaces/dapr/quickstarts/pub-sub/perl-subscriber$ umask 0002
builder@DESKTOP-JBA79RT:~/Workspaces/dapr/quickstarts/pub-sub/perl-subscriber$ sudo cpan install HTTP::Server::Simple::CGI JSON
…
Installing /usr/local/bin/json_xs
Appending installation info to /usr/local/lib/x86_64-linux-gnu/perl/5.26.1/perllocal.pod
MLEHMANN/JSON-XS-4.03.tar.gz
/usr/bin/make install -- OK
To actually launch this, we'll need a Dockerfile:
FROM debian:stable-slim AS base
LABEL maintainer="Isaac Johnson <isaac.johnson@gmail.com>"
# Credit to "Micheal Waltz <dockerfiles@ecliptik.com>"
# https://www.ecliptik.com/Containerizing-a-Perl-Script/
# Environment
ENV DEBIAN_FRONTEND=noninteractive \
LANG=en_US.UTF-8 \
LC_ALL=C.UTF-8 \
LANGUAGE=en_US.UTF-8
# Install runtime packages
RUN apt-get update \
&& apt-get install -y \
perl
# Set app dir
WORKDIR /app
# Intermediate build layer
FROM base AS build
#Update system and install packages
RUN apt-get update \
&& apt-get install -yq \
build-essential \
cpanminus
# Install cpan modules
RUN cpanm JSON HTTP::Server::Simple::CGI HTML::Entities Data::Dumper
# Runtime layer
FROM base AS run
# Copy build artifacts from build layer
COPY --from=build /usr/local /usr/local
# Copy perl script
COPY ./server.pl .
RUN chmod 755 ./server.pl
# Set Entrypoint
#ENTRYPOINT ["sleep"]
#CMD ["1000"]
ENTRYPOINT [ "/app/server.pl" ]One thing you’ll see is the ENTRYPOINT/CMD section at the bottom. Sometimes it can be hard to debug a microservice when you cannot see the running files. This way we can sleep the container and interactively login with kubectl exec -it (pod) -- /bin/bash and see what any issues are.
In my case, in debugging i found the missing HTML::Entities (which for some reason was not needed locally) was causing the script to crash.
Now that we have a Dockerfile we can build it
$ docker build -t dapr-perl .
[+] Building 155.6s (12/12) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 945B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/debian:stable-slim 15.0s
=> [internal] load build context 0.1s
=> => transferring context: 2.13kB 0.0s
=> [base 1/3] FROM docker.io/library/debian:stable-slim@sha256:f31a1f9e274ebf9a97b964c9aed3386e1720d56a3d2c74aaa6ea25f3545fae2e 4.7s
=> => resolve docker.io/library/debian:stable-slim@sha256:f31a1f9e274ebf9a97b964c9aed3386e1720d56a3d2c74aaa6ea25f3545fae2e 0.0s
=> => sha256:f052c451335940bea31ab0e58feff9fc657959c49150be8ed3d4dd592790bb3b 1.46kB / 1.46kB 0.0s
=> => sha256:7fa62122c34635d215ecb61e3d515d9eb1676a0f258d352e08c13da90f93ad19 27.14MB / 27.14MB 2.6s
=> => sha256:f31a1f9e274ebf9a97b964c9aed3386e1720d56a3d2c74aaa6ea25f3545fae2e 1.85kB / 1.85kB 0.0s
=> => sha256:65c240929acadafb6cdf7447d7e3ba1abc214a8b008fe4e256556f38647bec5b 529B / 529B 0.0s
=> => extracting sha256:7fa62122c34635d215ecb61e3d515d9eb1676a0f258d352e08c13da90f93ad19 1.6s
=> [base 2/3] RUN apt-get update && apt-get install -y perl 7.2s
=> [base 3/3] WORKDIR /app 0.1s
=> [build 1/2] RUN apt-get update && apt-get install -yq build-essential cpanminus 40.7s
=> [build 2/2] RUN umask 0002 && cpanm JSON HTTP::Server::Simple::CGI 86.5s
=> [run 1/2] COPY --from=build /usr/local /usr/local 0.1s
=> [run 2/2] COPY ./server.pl . 0.1s
=> exporting to image
0.7s
=> => exporting layers 0.7s
=> => writing image sha256:ca5ee9dc4c6a6ff7db61e53c338371a9b791fef3b4b8d47b70d30d2ab6c45f15 0.0s
=> => naming to docker.io/library/dapr-perl
We can also build with a versioned tag so we can use this in K3s
$ docker build -t dapr-perl:v12 .
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 38B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/debian:stable-slim 16.3s
=> [base 1/3] FROM docker.io/library/debian:stable-slim@sha256:f31a1f9e274ebf9a97b964c9aed3386e1720d56a3d2c74aaa6ea25f3545fae2e 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 3.00kB 0.0s
=> CACHED [base 2/3] RUN apt-get update && apt-get install -y perl 0.0s
=> CACHED [base 3/3] WORKDIR /app 0.0s
=> CACHED [build 1/2] RUN apt-get update && apt-get install -yq build-essential cpanminus 0.0s
=> CACHED [build 2/2] RUN cpanm JSON HTTP::Server::Simple::CGI HTML::Entities Data::Dumper 0.0s
=> CACHED [run 1/3] COPY --from=build /usr/local /usr/local 0.0s
=> [run 2/3] COPY ./server.pl . 0.1s
=> [run 3/3] RUN chmod 755 ./server.pl 0.5s
=> exporting to image 0.2s
=> => exporting layers 0.1s
=> => writing image sha256:22f1c814859b095a8b5839ba3fcdde8f2f03ad8319fe8aa87ea09811e7f58d30 0.0s
=> => naming to docker.io/library/dapr-perl:v12
Find the image
$ docker images | grep v12 | sed 's/.* v12\s*//' | sed 's/ .*$//' | head -n1 | tr -d '\n'
22f1c814859b
Tag it with our dockerhub URL
$ docker tag $(docker images | grep v12 | sed 's/.* v12\s*//' |//' | head -n1 | tr -d '\n') idjohnson/dapr-perl:v12
Then push it to dockerhub
$ docker push idjohnson/dapr-perl:v12
The push refers to repository [docker.io/idjohnson/dapr-perl]
aa49ecb00de7: Pushed
5a63cb8b3f85: Pushed
9abc3987cd18: Layer already exists
35038a85f076: Layer already exists
e547710c9503: Layer already exists
ff87a6345676: Layer already exists
v12: digest: sha256:3f240325ac929b12cb2a51ed237f67d8b71463203b8445d557e45c9bba1484e4 size: 1574
Lastly, we can replace the version in our YAML and apply it (deploy it to our local K3S)
$ sed -i 's/dapr-perl:v.*/dapr-perl:v12/' ../deploy/perl-subscriber.yaml
$ cat ../deploy/perl-subscriber.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: perl-subscriber
labels:
app: perl-subscriber
spec:
replicas: 1
selector:
matchLabels:
app: perl-subscriber
template:
metadata:
labels:
app: perl-subscriber
annotations:
dapr.io/enabled: "true"
dapr.io/app-id: "perl-subscriber"
dapr.io/app-port: "8080"
spec:
containers:
- name: perl-subscriber
image: idjohnson/dapr-perl:v12
ports:
- containerPort: 8080
imagePullPolicy: Always
$ kubectl apply -f ../deploy/perl-subscriber.yaml
deployment.apps/perl-subscriber configured
Verification
To send messages, we can use the react form already deployed:
$ kubectl port-forward react-form-7975b5fff9-db8ww 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080
Handling connection for 8080
Let’s go ahead and watch the logs of the pod to see what happens (and bear with me, first time adding an animated GIF to show a process in a blog entry):
Continuing improvements
I made changes enough to start to just have 2 lines of bash to build and deploy:
builder@DESKTOP-JBA79RT:~/Workspaces/dapr/quickstarts/pub-sub/perl-subscriber$ docker build -t dapr-perl:v13 . && docker tag $(docker images | grep v13 | sed 's/.* v13\s*//' | sed 's/ .*$//' | head -n1 | tr -d '\n') idjohnson/dapr-perl:v13 && docker push idjohnson/dapr-perl:v13
[+] Building 17.7s (13/13) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 38B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/debian:stable-slim 16.4s
=> [base 1/3] FROM docker.io/library/debian:stable-slim@sha256:f31a1f9e274ebf9a97b964c9aed3386e1720d56a3d2c74aaa6ea25f3545fae2e 0.0s
=> [internal] load build context 0.1s
=> => transferring context: 3.03kB 0.0s
=> CACHED [base 2/3] RUN apt-get update && apt-get install -y perl 0.0s
=> CACHED [base 3/3] WORKDIR /app 0.0s
=> CACHED [build 1/2] RUN apt-get update && apt-get install -yq build-essential cpanminus 0.0s
=> CACHED [build 2/2] RUN cpanm JSON HTTP::Server::Simple::CGI HTML::Entities Data::Dumper 0.0s
=> CACHED [run 1/3] COPY --from=build /usr/local /usr/local 0.0s
=> [run 2/3] COPY ./server.pl . 0.1s
=> [run 3/3] RUN chmod 755 ./server.pl 0.7s
=> exporting to image 0.2s
=> => exporting layers 0.1s
=> => writing image sha256:6f5419e726dc45351c7c62fc110a21212163b705f5a35605d2db49c4f756cd8e 0.0s
=> => naming to docker.io/library/dapr-perl:v13 0.0s
The push refers to repository [docker.io/idjohnson/dapr-perl]
e42bd628e149: Layer already exists
ac53dab07bb3: Layer already exists
35038a85f076: Layer already exists
e547710c9503: Layer already exists
ff87a6345676: Layer already exists
v13: digest: sha256:4a2f5f35fe4425d400619643dd70d8870dcb995a4df3a46175edb12dc697a895 size: 1365
And then to deploy
builder@DESKTOP-JBA79RT:~/Workspaces/dapr/quickstarts/pub-sub/perl-subscriber$ sed -i 's/dapr-perl:v.*/dapr-perl:v13/' ../deploy/perl-subscriber.yaml && kubectl apply -f ../deploy/perl-subscriber.yaml
deployment.apps/perl-subscriber configured
With a little testing, i worked out the dumper logic in the A handler. Now when testing, we can see the proper structured cloud event output:
Which renders this:
$decdata = {
'data' => {
'message' => 'Testing',
'messageType' => 'A'
},
'datacontenttype' => 'application/json',
'specversion' => '1.0',
'id' => '7683ec76-5bf2-4575-852e-e24b9cefd841',
'source' => 'react-form',
'topic' => 'A',
'pubsubname' => 'pubsub',
'type' => 'com.dapr.event.sent',
'traceid' => '00-1701e80ffda1590595abda90202cb7aa-1bc556cc0e5abfbc-00'
};
{ 'data' => { 'message' => 'Testing', 'messageType' => 'A' }, 'datacontenttype' => 'application/json', 'specversion' => '1.0', 'id' => '7683ec76-5bf2-4575-852e-e24b9cefd841', 'source' => 'react-form', 'topic' => 'A', 'pubsubname' => 'pubsub', 'type' => 'com.dapr.event.sent', 'traceid' => '00-1701e80ffda1590595abda90202cb7aa-1bc556cc0e5abfbc-00' }
Triggering External Events
What could we do with this? Perhaps we could use pub-sub to alert a Teams channel:
In Microsoft Teams, go to connectors in a channel
Choose configure or Add on Webhook
We can create a “PubSub” one with the Perl camel
Which will give us a URL like this: https://princessking.webhook.office.com/webhookb2/ccef6213-8978-482a-85a0-4885a61efd4e@92a2e5a9-a8ab-4d8c-91eb-8af9712c16d5/IncomingWebhook/a36d5cf04a7f47289b814288d92d2f60/26a39c32-7dcf-48b5-b24c-adda8d65d0c4
Which we can test:
Testing changes in server.pl locally, we can curl to the running script fake cloudevent
$ curl -X POST http://localhost:8080/A -d '{ "data" : { "message" : "Testing", "messageType" : "A" }, "datacontenttype" : "application/json", "specversion" : "1.0"
, "id" : "7683ec76-5bf2-4575-852e-e24b9cefd841", "source" : "react-form", "topic" : "A", "pubsubname" : "pubsub", "type"
: "com.dapr.event.sent", "traceid" : "00-1701e80ffda1590595abda90202cb7aa-1bc556cc0e5abfbc-00" }' -H 'Content-Type: app
lication/json'
For instance, the resp_A set as:
sub resp_A {
my $cgi = shift; # CGI.pm object
return if !ref $cgi;
my $decdata = decode_json(scalar $cgi->param('POSTDATA'));
print $cgi->header('application/json');
my $dVal = Data::Dumper->new([$decdata], [qw(decdata *ary)]);
print STDERR "\n message: " . $decdata->{'data'}->{'message'} . "\n";
print STDERR "\n pubsubname: " . $decdata->{'pubsubname'} . "\n";
my $message = $decdata->{'data'}->{'message'};
$message =~ s/"/\\"/g; # escape the double quotes at least
my $cmd = "curl -H 'Content-Type: application/json' -d '{\"text\": \"$message\"}' https://princessking.webhook.office.com/webhookb2/ccef6213-8978-482a-85a0-4885a61efd4e\@92a2e5a9-a8ab-4d8c-91eb-8af9712c16d5/IncomingWebhook/a36d5cf04a7f47289b814288d92d2f60/26a39c32-7dcf-48b5-b24c-adda8d65d0c4";
print STDERR "\ncd,m: $cmd\n";
my $rc =`$cmd`;
print STDERR "\n$rc\n";
print STDERR $dVal->Dump;
print STDERR "\n";
}
(note: if you are following along with your own teams URL, don't forget to escape that "@".. that took me a while to notice in debugging)
This means we can trigger it
And see the result now in Teams
If we wanted to keep going with this, we would likely move the webhook URL into a secret and bring it back in.
$ echo "https://princessking.webhook.office.com/webhookb2/ccef6213-8978-482a-85a0-4885a61efd4e@92a2e5a9-a8ab-4d8c-91eb-8af9712c16d5/IncomingWebhook/a36d5cf04a7f472
89b814288d92d2f60/26a39c32-7dcf-48b5-b24c-adda8d65d0c4" > webhookURL
$ kubectl create secret generic teamshook -
-from-file=hookURL=./webhookURL
secret/teamshook created
and then verify it
$ kubectl get secret teamshook -o yaml
apiVersion: v1
data:
hookURL: aHR0cHM6Ly9wcmluY2Vzc2tpbmcud2ViaG9vay5vZmZpY2UuY29tL3dlYmhvb2tiMi9jY2VmNjIxMy04OTc4LTQ4MmEtODVhMC00ODg1YTYxZWZkNGVAOTJhMmU1YTktYThhYi00ZDhjLTkxZWItOGFmOTcxMmMxNmQ1L0luY29taW5nV2ViaG9vay9hMzZkNWNmMDRhN2Y0NzI4OWI4MTQyODhkOTJkMmY2MC8yNmEzOWMzMi03ZGNmLTQ4YjUtYjI0Yy1hZGRhOGQ2NWQwYzQK
kind: Secret
metadata:
creationTimestamp: "2021-04-03T14:38:32Z"
managedFields:
We can test locally:
$ export WEBHOOKURL=https://princessking.webhook.office.com/webhookb2/ccef6213-8978-482a-85a0-4885a61efd4e@92a2e5a9-a8ab-4d8c-91eb-8af9712c16d5/IncomingWebhook/a36d5cf04a7f47289b814288d92d2f60/26a39c32-7dcf-48b5-b24c-adda8d65d0c4
$ export | grep WEBHOOK
declare -x WEBHOOKURL="https://princessking.webhook.office.com/webhookb2/ccef6213-8978-482a-85a0-4885a61efd4e@92a2e5a9-a8ab-4d8c-91eb-8af9712c16d5/IncomingWebhook/a36d5cf04a7f47289b814288d92d2f60/26a39c32-7dcf-48b5-b24c-adda8d65d0c4"
In the server.pl, we can add a warning at the top if the env var is missing:
use Data::Dumper;
unless ($ENV{'WEBHOOKURL'})
{
print "WARNING: WEBHOOKURL env var not found. Will disable Teams notifications!\n";
print STDERR "WARNING: WEBHOOKURL env var not found. Will disable Teams notifications!\n";
}
$gDebug = 0;Then in the resp_A, add a block to use it for the curl command to teams:
if ($ENV{'WEBHOOKURL'})
{
my $cmd = "curl -H 'Content-Type: application/json' -d '{\"text\": \"$message\"}' $ENV{'WEBHOOKURL'}";
print STDERR "\ncd,m: $cmd\n";
my $rc =`$cmd`;
print STDERR "\n$rc\n";
}
And we can see it now
And reflected in Teams here:
Lastly, to use this in Kubernetes, we will need to update the YAML file:
apiVersion: apps/v1
kind: Deployment
metadata:
name: perl-subscriber
labels:
app: perl-subscriber
spec:
replicas: 1
selector:
matchLabels:
app: perl-subscriber
template:
metadata:
labels:
app: perl-subscriber
annotations:
dapr.io/enabled: "true"
dapr.io/app-id: "perl-subscriber"
dapr.io/app-port: "8080"
spec:
containers:
- name: perl-subscriber
image: idjohnson/dapr-perl:v15
env:
- name: WEBHOOKURL
valueFrom:
secretKeyRef:
name: teamshook
key: hookURL
ports:
- containerPort: 8080
imagePullPolicy: AlwaysFinally, the final versions (for now) of our perl-subscriber files.
The server.pl
#!/usr/bin/perl
{
package MyWebServer;
use HTTP::Server::Simple::CGI;
use base qw(HTTP::Server::Simple::CGI);
use JSON;
use Data::Dumper;
unless ($ENV{'WEBHOOKURL'})
{
print "WARNING: WEBHOOKURL env var not found. Will disable Teams notifications!\n";
print STDERR "WARNING: WEBHOOKURL env var not found. Will disable Teams notifications!\n";
}
$gDebug = 0;
my %dispatch = (
'/dapr/subscribe' => \&resp_subscribe,
'/hello' => \&resp_hello,
'/A' => \&resp_A,
'/B' => \&resp_B,
'/C' => \&resp_C,
# ...
);
sub handle_request {
my $self = shift;
my $cgi = shift;
my $path = $cgi->path_info();
my $handler = $dispatch{$path};
if (ref($handler) eq "CODE") {
print "HTTP/1.0 200 OK\r\n";
$handler->($cgi);
} else {
print "HTTP/1.0 404 Not found\r\n";
print $cgi->header,
$cgi->start_html('Not found'),
$cgi->h1('Not found'),
$cgi->end_html;
}
}
sub resp_hello {
my $cgi = shift; # CGI.pm object
return if !ref $cgi;
my $who = $cgi->param('name');
print $cgi->header,
$cgi->start_html("Hello"),
$cgi->h1("Hello $who!"),
$cgi->end_html;
}
sub resp_A {
my $cgi = shift; # CGI.pm object
return if !ref $cgi;
my $decdata = decode_json(scalar $cgi->param('POSTDATA'));
print $cgi->header('application/json');
my $dVal = Data::Dumper->new([$decdata], [qw(decdata *ary)]);
print STDERR "\n message: " . $decdata->{'data'}->{'message'} . "\n";
print STDERR "\n pubsubname: " . $decdata->{'pubsubname'} . "\n";
my $message = $decdata->{'data'}->{'message'};
$message =~ s/"/\\"/g; # escape the double quotes at least
if ($ENV{'WEBHOOKURL'})
{
my $cmd = "curl -H 'Content-Type: application/json' -d '{\"text\": \"$message\"}' $ENV{'WEBHOOKURL'}";
print STDERR "\ncd,m: $cmd\n";
my $rc =`$cmd`;
print STDERR "\n$rc\n";
}
print STDERR $dVal->Dump;
print STDERR "\n";
}
sub resp_B {
my $cgi = shift; # CGI.pm object
return if !ref $cgi;
print $cgi->header('application/json');
print STDERR $cgi->Dump;
}
sub resp_C {
my $cgi = shift; # CGI.pm object
return if !ref $cgi;
print $cgi->header('application/json');
if ($cgi->param('POSTDATA'))
{
print STDERR $cgi->param('POSTDATA');
} else {
print STDERR "C: NO VAR\n";
}
print STDERR "\n";
}
sub resp_subscribe {
my $cgi = shift; # CGI.pm object
print $cgi->header('application/json');
my @rec_hash = ( { 'pubsubname' => 'pubsub', 'topic' => 'A', 'route' => 'A' }, { 'pubsubname' => 'pubsub', 'topic' => 'B', 'route' => 'B'}, { 'pubsubname' => 'pubsub', 'topic' => 'C', 'route' => 'C'} );
my $json = encode_json \@rec_hash;
print "$json\n";
}
}
# start the server on port 8080
if ($gDebug > 0) {
my $pid = MyWebServer->new(8080)->background();
print "Use 'kill $pid' to stop server.\n";
} else {
print "running on 8080\n";
my $server = MyWebServer->new(8080);
$server->run();
}
And Dockerfile:
FROM debian:stable-slim AS base
LABEL maintainer="Isaac Johnson <isaac.johnson@gmail.com>"
# Credit to "Micheal Waltz <dockerfiles@ecliptik.com>"
# https://www.ecliptik.com/Containerizing-a-Perl-Script/
# Environment
ENV DEBIAN_FRONTEND=noninteractive \
LANG=en_US.UTF-8 \
LC_ALL=C.UTF-8 \
LANGUAGE=en_US.UTF-8
# Install runtime packages
RUN apt-get update \
&& apt-get install -y \
perl
# Set app dir
WORKDIR /app
# Intermediate build layer
FROM base AS build
#Update system and install packages
RUN apt-get update \
&& apt-get install -yq \
build-essential \
cpanminus
# Install cpan modules
RUN cpanm JSON HTTP::Server::Simple::CGI HTML::Entities Data::Dumper
# Runtime layer
FROM base AS run
# Copy build artifacts from build layer
COPY --from=build /usr/local /usr/local
# Copy perl script
COPY ./server.pl .
RUN chmod 755 ./server.pl
# Set Entrypoint
#ENTRYPOINT ["sleep"]
#CMD ["1000"]
ENTRYPOINT [ "/app/server.pl" ]Summary
We took the basic pub sub model that we followed in the first guide and worked it a bit more. We deployed locally and this time to a local on-prem K3S cluster. We worked out changes locally and then extended some functionality to pass forward a dapr notification to a Teams channel.
I plan to dig into other Dapr features beyond the pub/sub in future blogs, but for now, this is a really handy bit of functionality I plan to use in my microservices.
