Published: Apr 16, 2024 by Isaac Johnson
We spoke last week about Redis (the company) abandoning Open-Source licensing for it’s product, Redis (the software). I first heard about Valkey, the larger of the new forks, from The Stack which pointed to that initial fork that was at https://github.com/placeholderkv/placeholderkv (now redirects to Valkey).
On Valkey
The article credits AWS’s Madelyn Olson, a primary Redis software contributor and AliCloud’s Zhou Zhou as the co-creators of the fork. After talking to the displeasure of the license change, Olson commented on Twitter:
“We had a really exciting community roadmap before Redis changed the license. Clustering improvements, performance improvements, triggers, and so much more. We’ll keep driving this forward”
I don’t use “twitter”, but on Mastodon, we can see growing support. @peterzaitsev said “I was excited to see it just took days. It was like, boom: ‘Redis, you choose to [mess with] the community, then Linux Foundation stands behind community.’ I think that was wonderful.” in reference to The Register reporting support from the Linux Foundation.
I also saw Kyle Davis just presented on Valkey at TexasLinuxFest. I would expect more like these in the future.
Since it’s open-source and fresh, let’s dig in and see how well it works.
Installation
Let’s start simple. Let us pull the code and attempt a local compile.
I’ll clone the latest
builder@LuiGi:~/Workspaces$ git clone https://github.com/valkey-io/valkey.git
Cloning into 'valkey'...
remote: Enumerating objects: 92809, done.
remote: Counting objects: 100% (787/787), done.
remote: Compressing objects: 100% (409/409), done.
remote: Total 92809 (delta 498), reused 597 (delta 375), pack-reused 92022
Receiving objects: 100% (92809/92809), 134.68 MiB | 9.06 MiB/s, done.
Resolving deltas: 100% (67779/67779), done.
builder@LuiGi:~/Workspaces$ cd valkey/
Then just run make
builder@LuiGi:~/Workspaces/valkey$ make
cd src && make all
make[1]: Entering directory '/home/builder/Workspaces/valkey/src'
/bin/sh: 1: pkg-config: not found
/bin/sh: 1: pkg-config: not found
/bin/sh: 1: pkg-config: not found
CC Makefile.dep
/bin/sh: 1: pkg-config: not found
/bin/sh: 1: pkg-config: not found
/bin/sh: 1: pkg-config: not found
rm -rf valkey-server valkey-sentinel valkey-cli valkey-benchmark valkey-check-rdb valkey-check-aof *.o *.gcda *.gcno *.gcov redis.info lcov-html Makefile.dep *.so
rm -f threads_mngr.d adlist.d quicklist.d ae.d anet.d dict.d kvstore.d server.d sds.d zmalloc.d lzf_c.d lzf_d.d pqsort.d zipmap.d sha1.d ziplist.d release.d networking.d util.d object.d db.d replication.d rdb.d t_string.d t_list.d t_set.d t_zset.d t_hash.d config.d aof.d pubsub.d multi.d debug.d sort.d intset.d syncio.d cluster.d cluster_legacy.d crc16.d endianconv.d slowlog.d eval.d bio.d rio.d rand.d memtest.d syscheck.d crcspeed.d crc64.d bitops.d sentinel.d notify.d setproctitle.d blocked.d hyperloglog.d latency.d sparkline.d valkey-check-rdb.d valkey-check-aof.d geo.d lazyfree.d module.d evict.d expire.d geohash.d geohash_helper.d childinfo.d defrag.d siphash.d rax.d t_stream.d listpack.d localtime.d lolwut.d lolwut5.d lolwut6.d acl.d tracking.d socket.d tls.d sha256.d timeout.d setcpuaffinity.d monotonic.d mt19937-64.d resp_parser.d call_reply.d script_lua.d script.d functions.d function_lua.d commands.d strl.d connection.d unix.d logreqres.d anet.d adlist.d dict.d valkey-cli.d zmalloc.d release.d ae.d serverassert.d crcspeed.d crc64.d siphash.d crc16.d monotonic.d cli_common.d mt19937-64.d strl.d cli_commands.d ae.d anet.d valkey-benchmark.d adlist.d dict.d zmalloc.d serverassert.d release.d crcspeed.d crc64.d siphash.d crc16.d monotonic.d cli_common.d mt19937-64.d strl.d
(cd ../deps && make distclean)
make[2]: Entering directory '/home/builder/Workspaces/valkey/deps'
(cd hiredis && make clean) > /dev/null || true
(cd linenoise && make clean) > /dev/null || true
(cd lua && make clean) > /dev/null || true
(cd jemalloc && [ -f Makefile ] && make distclean) > /dev/null || true
(cd hdr_histogram && make clean) > /dev/null || true
(cd fpconv && make clean) > /dev/null || true
(rm -f .make-*)
make[2]: Leaving directory '/home/builder/Workspaces/valkey/deps'
(cd modules && make clean)
make[2]: Entering directory '/home/builder/Workspaces/valkey/src/modules'
rm -rf *.xo *.so
make[2]: Leaving directory '/home/builder/Workspaces/valkey/src/modules'
(cd ../tests/modules && make clean)
... snip ...
Processing json files...
Linking container command to subcommands...
Checking all commands...
Generating commands.def...
All done, exiting.
CC commands.o
CC strl.o
CC connection.o
CC unix.o
CC logreqres.o
LINK valkey-server
INSTALL valkey-sentinel
CC valkey-cli.o
CC serverassert.o
CC cli_common.o
CC cli_commands.o
LINK valkey-cli
CC valkey-benchmark.o
LINK valkey-benchmark
INSTALL valkey-check-rdb
INSTALL valkey-check-aof
Hint: It's a good idea to run 'make test' ;)
make[1]: Leaving directory '/home/builder/Workspaces/valkey/src'
Now if we want SSL support, we need to ensure we have libssl-dev installed
$ sudo apt-get install libssl-dev
Then can make with SSL support
builder@LuiGi:~/Workspaces/valkey$ make distclean
builder@LuiGi:~/Workspaces/valkey$ make BUILD_TLS=yes
cd src && make all
make[1]: Entering directory '/home/builder/Workspaces/valkey/src'
/bin/sh: 1: pkg-config: not found
/bin/sh: 1: pkg-config: not found
/bin/sh: 1: pkg-config: not found
CC Makefile.dep
/bin/sh: 1: pkg-config: not found
/bin/sh: 1: pkg-config: not found
/bin/sh: 1: pkg-config: not found
rm -rf valkey-server valkey-sentinel valkey-cli valkey-benchmark valkey-check-rdb valkey-check-aof *.o *.gcda *.gcno *.gcov redis.info lcov-html Makefile.dep *.so
rm -f threads_mngr.d adlist.d quicklist.d ae.d anet.d dict.d kvstore.d server.d sds.d zmalloc.d lzf_c.d lzf_d.d pqsort.d zipmap.d sha1.d ziplist.d release.d networking.d util.d object.d db.d replication.d rdb.d t_string.d t_list.d t_set.d t_zset.d t_hash.d config.d aof.d pubsub.d multi.d debug.d sort.d intset.d syncio.d cluster.d cluster_legacy.d crc16.d endianconv.d slowlog.d eval.d bio.d rio.d rand.d memtest.d syscheck.d crcspeed.d crc64.d bitops.d sentinel.d notify.d setproctitle.d blocked.d hyperloglog.d latency.d sparkline.d valkey-check-rdb.d valkey-check-aof.d geo.d lazyfree.d module.d evict.d expire.d geohash.d geohash_helper.d childinfo.d defrag.d siphash.d rax.d t_stream.d listpack.d localtime.d lolwut.d lolwut5.d lolwut6.d acl.d tracking.d socket.d tls.d sha256.d timeout.d setcpuaffinity.d monotonic.d mt19937-64.d resp_parser.d call_reply.d script_lua.d script.d functions.d function_lua.d commands.d strl.d connection.d unix.d logreqres.d anet.d adlist.d dict.d valkey-cli.d zmalloc.d release.d ae.d serverassert.d crcspeed.d crc64.d siphash.d crc16.d monotonic.d cli_common.d mt19937-64.d strl.d cli_commands.d ae.d anet.d valkey-benchmark.d adlist.d dict.d zmalloc.d serverassert.d release.d crcspeed.d crc64.d siphash.d crc16.d monotonic.d cli_common.d mt19937-64.d strl.d
(cd ../deps && make distclean)
... snip ...
LINK valkey-server
INSTALL valkey-sentinel
CC valkey-cli.o
CC serverassert.o
CC cli_common.o
CC cli_commands.o
LINK valkey-cli
CC valkey-benchmark.o
LINK valkey-benchmark
INSTALL valkey-check-rdb
INSTALL valkey-check-aof
Hint: It's a good idea to run 'make test' ;)
make[1]: Leaving directory '/home/builder/Workspaces/valkey/src'
Let’s run some Unit Tests
builder@LuiGi:~/Workspaces/valkey$ ./utils/gen-test-certs.sh
Certificate request self-signature ok
subject=O = Redis Test, CN = Server-only
Certificate request self-signature ok
subject=O = Redis Test, CN = Client-only
Certificate request self-signature ok
subject=O = Redis Test, CN = Generic-cert
Generating DH parameters, 2048 bit long safe prime
....+.........................................................................................................+.+.....................+...............+...........................................................................+................................................................................................................................................................................................................................+.............+..........................+..................................................................................................................................................................................................................................+.............................................................................................................................................................................................................+...............................................................................+........................................................................+..................................................................................................................................................................................+...................................+.........................................................................................+.............................................+............+...........................................................................................................................................................................................+.................................+....................................................................+...+........................................................................................+......................................................................................................+......+..................................................................+..........+..+.................................................................................................................+.......................................................................................+...................................................................................................................................................................................................................................................................................................................................................+..........+..................................+..................+................................................................................................+.................................................................+....................................+............................................
...........++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*++*
I’ll add TCL so i can run tests
builder@LuiGi:~/Workspaces/valkey$ sudo apt-get install -y tcl
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
libtcl8.6 tcl8.6
Suggested packages:
tcl-tclreadline
The following NEW packages will be installed:
libtcl8.6 tcl tcl8.6
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 1009 kB of archives.
After this operation, 4262 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 libtcl8.6 amd64 8.6.12+dfsg-1build1 [990 kB]
Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB]
Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4678 B]
Fetched 1009 kB in 5s (184 kB/s)
Selecting previously unselected package libtcl8.6:amd64.
(Reading database ... 81862 files and directories currently installed.)
Preparing to unpack .../libtcl8.6_8.6.12+dfsg-1build1_amd64.deb ...
Unpacking libtcl8.6:amd64 (8.6.12+dfsg-1build1) ...
Selecting previously unselected package tcl8.6.
Preparing to unpack .../tcl8.6_8.6.12+dfsg-1build1_amd64.deb ...
Unpacking tcl8.6 (8.6.12+dfsg-1build1) ...
Selecting previously unselected package tcl.
Preparing to unpack .../tcl_8.6.11+1build2_amd64.deb ...
Unpacking tcl (8.6.11+1build2) ...
Setting up libtcl8.6:amd64 (8.6.12+dfsg-1build1) ...
Setting up tcl8.6 (8.6.12+dfsg-1build1) ...
Setting up tcl (8.6.11+1build2) ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for libc-bin (2.35-0ubuntu3.6) ...
builder@LuiGi:~/Workspaces/valkey$ sudo apt-get install -y tcl tcl-tls
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
tcl is already the newest version (8.6.11+1build2).
The following NEW packages will be installed:
tcl-tls
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 68.2 kB of archives.
After this operation, 171 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 tcl-tls amd64 1.7.22-2ubuntu1 [68.2 kB]
Fetched 68.2 kB in 1s (101 kB/s)
Selecting previously unselected package tcl-tls:amd64.
(Reading database ... 82117 files and directories currently installed.)
Preparing to unpack .../tcl-tls_1.7.22-2ubuntu1_amd64.deb ...
Unpacking tcl-tls:amd64 (1.7.22-2ubuntu1) ...
Setting up tcl-tls:amd64 (1.7.22-2ubuntu1) ...
I ran tests.. way too many to show inline
$ ./runtest --tls
I can now run the server
builder@LuiGi:~/Workspaces/valkey/src$ ./valkey-server
3806:C 04 Apr 2024 18:34:35.749 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
3806:C 04 Apr 2024 18:34:35.749 * valkey version=255.255.255, bits=64, commit=bb1a3fff, modified=0, pid=3806, just started
3806:C 04 Apr 2024 18:34:35.749 # Warning: no config file specified, using the default config. In order to specify a config file use ./valkey-server /path/to/redis.conf
3806:M 04 Apr 2024 18:34:35.750 * Increased maximum number of open files to 10032 (it was originally set to 1024).
3806:M 04 Apr 2024 18:34:35.750 * monotonic clock: POSIX clock_gettime
___ ___
( ) ( )
___ ___ .---. | | | | ___ .--. ___ ___
( )( ) / .-, \ | | | | ( ) / \ ( )( ) Valkey 255.255.255 (bb1a3fff/0) 64 bit
| | | | (__) ; | | | | | ' / | .-. ; | | | |
| | | | .'` | | | | |,' / | | | | | | | | Running in standalone mode
| | | | / .'| | | | | . '. | |/ | | ' | | Port: 6379
| | | | | / | | | | | | `. \ | ' _.' ' `-' | PID: 3806
' ' ; ' ; | ; | | | | | \ \ | .'.-. `.__. |
\ `' / ' `-' | | | | | \ . ' `-' / ___ | |
\__/ `.__.'_. (___) (___ ) (___) `.__.' ( )' | https://valkey.io
; `-' '
.__.'
3806:M 04 Apr 2024 18:34:35.750 * Server initialized
3806:M 04 Apr 2024 18:34:35.750 * Ready to accept connections tcp
I can now test with redis-cli
Container
I’ll be using the current https://github.com/valkey-io/valkey-container.git repo for these next steps.
It contains the Valkey container template that will be using Bashbrew.
Dependencies
First, I’ll need bashbrew.
At first, I though i might get that thru homebrew. homebrew setup in Linux
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
builder@LuiGi:~/Workspaces/valkey-container$ brew install bashbrew
Command 'brew' not found, did you mean:
command 'brec' from deb bplay (0.991-10build1)
command 'qbrew' from deb qbrew (0.4.1-8build1)
Try: sudo apt install <deb name>
Okay, i guess i have to download from Jenkins: https://doi-janky.infosiftr.net/job/bashbrew/job/master/lastSuccessfulBuild/artifact/
I copied it from my Downloads to my WSL instance
builder@LuiGi:~/Workspaces/valkey-container$ sudo cp /mnt/c/Users/isaac/Downloads/bashbrew-amd64 /usr/local/bin/bashbrew
[sudo] password for builder:
builder@LuiGi:~/Workspaces/valkey-container$ sudo chmod 755 /usr/local/bin/bashbrew
builder@LuiGi:~/Workspaces/valkey-container$ bashbrew -h
NAME:
bashbrew - canonical build tool for the official images
USAGE:
bashbrew [global options] command [command options] [arguments...]
VERSION:
v0.1.13
COMMANDS:
list, ls list repo:tag combinations for a given repo
build build (and tag) repo:tag combinations for a given repo
tag tag repo:tag into a namespace (especially for pushing)
push push namespace/repo:tag (see also "tag")
put-shared update shared tags in the registry (and multi-architecture tags)
help, h Shows a list of commands or help for one command
Turning Templates to Dockerfiles
I used apply-templates to create some unstable templates
builder@LuiGi:~/Workspaces/valkey-container$ ./apply-templates.sh
processing unstable/debian ...
processing unstable/alpine ...
builder@LuiGi:~/Workspaces/valkey-container$ ls
Dockerfile.template LICENSE README.md apply-templates.sh docker-entrypoint.sh generate-stackbrew-library.sh unstable update.sh versions.json versions.sh
builder@LuiGi:~/Workspaces/valkey-container$ cat unstable/
alpine/ debian/
builder@LuiGi:~/Workspaces/valkey-container$ cat unstable/alpine/
Dockerfile docker-entrypoint.sh
builder@LuiGi:~/Workspaces/valkey-container$ cat unstable/alpine/
If there was any doubt, we can cat the end of the file
builder@LuiGi:~/Workspaces/valkey-container/unstable/alpine$ cat Dockerfile | tail -n 5
COPY docker-entrypoint.sh /usr/local/bin/
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 6379
CMD ["valkey-server"]
However, initially, a Docker build died:
4.140 '/usr/local/bin/gosu' saved
4.140 + echo 'bbc4136d03ab138b1ad66fa4fc051bafc6cc7ffae632b069a53657279a450de3 */usr/local/bin/gosu'
4.141 + sha256sum -c -
4.151 /usr/local/bin/gosu: OK
4.151 + mktemp -d
4.152 + export 'GNUPGHOME=/tmp/tmp.fOmBkB'
4.152 + gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4
4.153 gpg: keybox '/tmp/tmp.fOmBkB/pubring.kbx' created
14.19 gpg: keyserver receive failed: Server indicated a failure
------
Dockerfile:25
--------------------
24 | ENV GOSU_VERSION 1.17
25 | >>> RUN set -eux; \
26 | >>> apk add --no-cache --virtual .gosu-fetch gnupg; \
27 | >>> arch="$(apk --print-arch)"; \
28 | >>> case "$arch" in \
29 | >>> 'x86_64') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-amd64'; sha256='bbc4136d03ab138b1ad66fa4fc051bafc6cc7ffae632b069a53657279a450de3' ;; \
30 | >>> 'aarch64') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-arm64'; sha256='c3805a85d17f4454c23d7059bcb97e1ec1af272b90126e79ed002342de08389b' ;; \
31 | >>> 'armhf') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-armhf'; sha256='e5866286277ff2a2159fb9196fea13e0a59d3f1091ea46ddb985160b94b6841b' ;; \
32 | >>> 'x86') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-i386'; sha256='087dbb8fe479537e64f9c86fa49ff3b41dee1cbd28739a19aaef83dc8186b1ca' ;; \
33 | >>> 'ppc64le') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-ppc64el'; sha256='1891acdcfa70046818ab6ed3c52b9d42fa10fbb7b340eb429c8c7849691dbd76' ;; \
34 | >>> 'riscv64') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-riscv64'; sha256='38a6444b57adce135c42d5a3689f616fc7803ddc7a07ff6f946f2ebc67a26ba6' ;; \
35 | >>> 's390x') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-s390x'; sha256='69873bab588192f760547ca1f75b27cfcf106e9f7403fee6fd0600bc914979d0' ;; \
36 | >>> 'armv7') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-armhf'; sha256='e5866286277ff2a2159fb9196fea13e0a59d3f1091ea46ddb985160b94b6841b' ;; \
37 | >>> *) echo >&2 "error: unsupported gosu architecture: '$arch'"; exit 1 ;; \
38 | >>> esac; \
39 | >>> wget -O /usr/local/bin/gosu.asc "$url.asc"; \
40 | >>> wget -O /usr/local/bin/gosu "$url"; \
41 | >>> echo "$sha256 */usr/local/bin/gosu" | sha256sum -c -; \
42 | >>> export GNUPGHOME="$(mktemp -d)"; \
43 | >>> gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4; \
44 | >>> gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \
45 | >>> gpgconf --kill all; \
46 | >>> rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; \
47 | >>> apk del --no-network .gosu-fetch; \
48 | >>> chmod +x /usr/local/bin/gosu; \
49 | >>> gosu --version; \
50 | >>> gosu nobody true
51 |
--------------------
ERROR: failed to solve: process "/bin/sh -c set -eux; \tapk add --no-cache --virtual .gosu-fetch gnupg; \tarch=\"$(apk --print-arch)\"; \tcase \"$arch\" in \t\t'x86_64') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-amd64'; sha256='bbc4136d03ab138b1ad66fa4fc051bafc6cc7ffae632b069a53657279a450de3' ;; \t\t'aarch64') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-arm64'; sha256='c3805a85d17f4454c23d7059bcb97e1ec1af272b90126e79ed002342de08389b' ;; \t\t'armhf') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-armhf'; sha256='e5866286277ff2a2159fb9196fea13e0a59d3f1091ea46ddb985160b94b6841b' ;; \t\t'x86') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-i386'; sha256='087dbb8fe479537e64f9c86fa49ff3b41dee1cbd28739a19aaef83dc8186b1ca' ;; \t\t'ppc64le') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-ppc64el'; sha256='1891acdcfa70046818ab6ed3c52b9d42fa10fbb7b340eb429c8c7849691dbd76' ;; \t\t'riscv64') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-riscv64'; sha256='38a6444b57adce135c42d5a3689f616fc7803ddc7a07ff6f946f2ebc67a26ba6' ;; \t\t's390x') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-s390x'; sha256='69873bab588192f760547ca1f75b27cfcf106e9f7403fee6fd0600bc914979d0' ;; \t\t'armv7') url='https://github.com/tianon/gosu/releases/download/1.17/gosu-armhf'; sha256='e5866286277ff2a2159fb9196fea13e0a59d3f1091ea46ddb985160b94b6841b' ;; \t\t*) echo >&2 \"error: unsupported gosu architecture: '$arch'\"; exit 1 ;; \tesac; \twget -O /usr/local/bin/gosu.asc \"$url.asc\"; \twget -O /usr/local/bin/gosu \"$url\"; \techo \"$sha256 */usr/local/bin/gosu\" | sha256sum -c -; \texport GNUPGHOME=\"$(mktemp -d)\"; \tgpg --batch --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4; \tgpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \tgpgconf --kill all; \trm -rf \"$GNUPGHOME\" /usr/local/bin/gosu.asc; \tapk del --no-network .gosu-fetch; \tchmod +x /usr/local/bin/gosu; \tgosu --version; \tgosu nobody true" did not complete successfully: exit code: 2
I got this error on Debian as well.
The fact is that I’m on a VPN and through a guest wifi presently. I’ll circle back and try at home off VPN next…
Working from my home wifi solved those issues:
$ docker build -t myvalkey-debian-test .
[+] Building 114.6s (14/14) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 7.59kB 0.0s
=> [internal] load metadata for docker.io/library/debian:bookworm-slim 0.8s
=> [auth] library/debian:pull token for registry-1.docker.io 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [1/8] FROM docker.io/library/debian:bookworm-slim@sha256:ccb33c3ac5b02588fc1d9e4fc09b952e433d0c54d8618d0ee1afadf1f3cf2455 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 42B 0.0s
=> CACHED [2/8] RUN set -eux; groupadd -r -g 999 valkey; useradd -r -g valkey -u 999 valkey 0.0s
=> CACHED [3/8] RUN set -eux; apt-get update; apt-get install -y --no-install-recommends tzdata ; rm -rf /var/lib/apt/lists/* 0.0s
=> [4/8] RUN set -eux; savedAptMark="$(apt-mark showmanual)"; apt-get update; apt-get install -y --no-install-recommends ca-certificates gnupg wget; rm -rf /var/lib/apt/lists/*; arch="$(dp 26.5s
=> [5/8] RUN set -eux; savedAptMark="$(apt-mark showmanual)"; apt-get update; apt-get install -y --no-install-recommends ca-certificates wget dpkg-dev gcc libc6-dev libssl-dev 85.8s
=> [6/8] RUN mkdir /data && chown valkey:valkey /data 0.6s
=> [7/8] WORKDIR /data 0.1s
=> [8/8] COPY docker-entrypoint.sh /usr/local/bin/ 0.1s
=> exporting to image 0.4s
=> => exporting layers 0.4s
=> => writing image sha256:4254910ab99b4444e6f554aa30627d7208aa0ae58f7c1c27122b3573a6513df0 0.0s
=> => naming to docker.io/library/myvalkey-debian-test 0.0s
What's Next?
View a summary of image vulnerabilities and recommendations → docker scout quickview
So I can see the images
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
myvalkey-debian-test latest 4254910ab99b About a minute ago 138MB
I can then test the image
Kubernetes
Let’s try using this in Kubernetes. To make it easy, I’ll push to Dockerhub.
builder@LuiGi:~/Workspaces/valkey-container/unstable/debian$ docker tag myvalkey-debian-test:latest idjohnson/myvalkey-server:latest
builder@LuiGi:~/Workspaces/valkey-container/unstable/debian$ docker push idjohnson/myvalkey-server:latest
The push refers to repository [docker.io/idjohnson/myvalkey-server]
de35904dc094: Pushed
5f70bf18a086: Pushed
e121f09e2c28: Pushed
0a690c4d99d1: Pushed
a7c6da9ddd24: Pushed
d8b470c15c5a: Pushed
6d157fc1f8ab: Pushed
a483da8ab3e9: Mounted from library/debian
latest: digest: sha256:af21cb4b35290034c45f638f09b148640f580575060f34ae5e7392216b68b09e size: 1987
Next, I’ll get the Bitnami helm charts
builder@LuiGi:~/Workspaces/valkey-container/unstable/debian$ helm repo add bitnami https://charts.bitnami.com/bitnami
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/builder/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/builder/.kube/config
"bitnami" has been added to your repositories
builder@LuiGi:~/Workspaces/valkey-container/unstable/debian$ helm repo update
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/builder/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/builder/.kube/config
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈
Now I’ll add it to my cluster with helm
$ kubectl create ns valkeytest
$ helm install my-redis -n valkeytest --set image.repository=idjohnson/myvalkey-server --set image.tag=latest bitnami/redis
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/builder/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/builder/.kube/config
NAME: my-redis
LAST DEPLOYED: Thu Apr 4 20:35:00 2024
NAMESPACE: valkeytest
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: redis
CHART VERSION: 19.0.2
APP VERSION: 7.2.4
** Please be patient while the chart is being deployed **
Redis® can be accessed on the following DNS names from within your cluster:
my-redis-master.valkeytest.svc.cluster.local for read/write operations (port 6379)
my-redis-replicas.valkeytest.svc.cluster.local for read-only operations (port 6379)
To get your password run:
export REDIS_PASSWORD=$(kubectl get secret --namespace valkeytest my-redis -o jsonpath="{.data.redis-password}" | base64 -d)
To connect to your Redis® server:
1. Run a Redis® pod that you can use as a client:
kubectl run --namespace valkeytest redis-client --restart='Never' --env REDIS_PASSWORD=$REDIS_PASSWORD --image docker.io/idjohnson/myvalkey-server:latest --command -- sleep infinity
Use the following command to attach to the pod:
kubectl exec --tty -i redis-client \
--namespace valkeytest -- bash
2. Connect using the Redis® CLI:
REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h my-redis-master
REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h my-redis-replicas
To connect to your database from outside the cluster execute the following commands:
kubectl port-forward --namespace valkeytest svc/my-redis-master 6379:6379 &
REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h 127.0.0.1 -p 6379
WARNING: There are "resources" sections in the chart not set. Using "resourcesPreset" is not recommended for production. For production installations, please set the following values according to your workload needs:
- master.resources
- replica.resources
+info https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
I then edited the statefulset to remove the cmd, args and redinessprobes:
$ kubectl get sts -n valkeytest my-redis-master -o yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
annotations:
meta.helm.sh/release-name: my-redis
meta.helm.sh/release-namespace: valkeytest
creationTimestamp: "2024-04-05T01:35:00Z"
generation: 3
labels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: my-redis
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: redis
app.kubernetes.io/version: 7.2.4
helm.sh/chart: redis-19.0.2
name: my-redis-master
namespace: valkeytest
resourceVersion: "5033502"
uid: bb30313c-9587-444f-8901-1e1db17a09b4
spec:
podManagementPolicy: OrderedReady
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: my-redis
app.kubernetes.io/name: redis
serviceName: my-redis-headless
template:
metadata:
annotations:
checksum/configmap: 86bcc953bb473748a3d3dc60b7c11f34e60c93519234d4c37f42e22ada559d47
checksum/health: aff24913d801436ea469d8d374b2ddb3ec4c43ee7ab24663d5f8ff1a1b6991a9
checksum/scripts: cdd24a5a5914d2c7dc654a082112588ef3e519477676882e617b92b536554aa6
checksum/secret: e226aa14c1d5c7c42bc7e76f1e761a7fd5a2a5f02c476cc44fb7cc32b81c3161
creationTimestamp: null
labels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: my-redis
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: redis
app.kubernetes.io/version: 7.2.4
helm.sh/chart: redis-19.0.2
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchLabels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: my-redis
app.kubernetes.io/name: redis
topologyKey: kubernetes.io/hostname
weight: 1
automountServiceAccountToken: false
containers:
- env:
- name: BITNAMI_DEBUG
value: "false"
- name: REDIS_REPLICATION_MODE
value: master
- name: ALLOW_EMPTY_PASSWORD
value: "no"
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
key: redis-password
name: my-redis
- name: REDIS_TLS_ENABLED
value: "no"
- name: REDIS_PORT
value: "6379"
image: docker.io/idjohnson/myvalkey-server:latest
imagePullPolicy: IfNotPresent
name: redis
ports:
- containerPort: 6379
name: redis
protocol: TCP
resources:
limits:
cpu: 150m
ephemeral-storage: 1Gi
memory: 192Mi
requests:
cpu: 100m
ephemeral-storage: 50Mi
memory: 128Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
runAsGroup: 1001
runAsNonRoot: true
runAsUser: 1001
seLinuxOptions: {}
seccompProfile:
type: RuntimeDefault
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /opt/bitnami/scripts/start-scripts
name: start-scripts
- mountPath: /health
name: health
- mountPath: /data
name: redis-data
- mountPath: /opt/bitnami/redis/mounted-etc
name: config
- mountPath: /opt/bitnami/redis/etc/
name: empty-dir
subPath: app-conf-dir
- mountPath: /tmp
name: empty-dir
subPath: tmp-dir
dnsPolicy: ClusterFirst
enableServiceLinks: true
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 1001
fsGroupChangePolicy: Always
serviceAccount: my-redis-master
serviceAccountName: my-redis-master
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 493
name: my-redis-scripts
name: start-scripts
- configMap:
defaultMode: 493
name: my-redis-health
name: health
- configMap:
defaultMode: 420
name: my-redis-configuration
name: config
- emptyDir: {}
name: empty-dir
updateStrategy:
type: RollingUpdate
volumeClaimTemplates:
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: null
labels:
app.kubernetes.io/component: master
app.kubernetes.io/instance: my-redis
app.kubernetes.io/name: redis
name: redis-data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 8Gi
volumeMode: Filesystem
status:
phase: Pending
status:
availableReplicas: 1
collisionCount: 0
currentReplicas: 1
currentRevision: my-redis-master-78f96d6f9
observedGeneration: 3
readyReplicas: 1
replicas: 1
updateRevision: my-redis-master-78f96d6f9
updatedReplicas: 1
We can now test
builder@LuiGi:~$ kubectl port-forward --namespace valkeytest svc/my-redis-master 6379:6379 &
[1] 60907
builder@LuiGi:~$ Forwarding from 127.0.0.1:6379 -> 6379
Forwarding from [::1]:6379 -> 6379
builder@LuiGi:~$ REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h 127.0.0.1 -p 6379
Handling connection for 6379
Warning: AUTH failed
127.0.0.1:6379> get test
(nil)
127.0.0.1:6379> set testkey val1
OK
127.0.0.1:6379> get testkey
"val1"
127.0.0.1:6379>
Repackaging
Let’s see if we can just regexp the chart to make the regular Bitnami Redis charts into Valkey ones
I’ll pull the latest Redis helm chart
builder@DESKTOP-QADGF36:~/Workspaces/valkey$ helm pull bitnami/redis
builder@DESKTOP-QADGF36:~/Workspaces/valkey$ ls
redis-19.0.1.tgz
builder@DESKTOP-QADGF36:~/Workspaces/valkey$ tar -xzvf redis-19.0.1.tgz
redis/Chart.yaml
redis/Chart.lock
redis/values.yaml
redis/values.schema.json
redis/templates/NOTES.txt
redis/templates/_helpers.tpl
redis/templates/configmap.yaml
redis/templates/extra-list.yaml
redis/templates/headless-svc.yaml
redis/templates/health-configmap.yaml
redis/templates/master/application.yaml
redis/templates/master/psp.yaml
redis/templates/master/pvc.yaml
redis/templates/master/service.yaml
redis/templates/master/serviceaccount.yaml
redis/templates/metrics-svc.yaml
redis/templates/networkpolicy.yaml
redis/templates/pdb.yaml
redis/templates/podmonitor.yaml
redis/templates/prometheusrule.yaml
redis/templates/replicas/application.yaml
redis/templates/replicas/hpa.yaml
redis/templates/replicas/service.yaml
redis/templates/replicas/serviceaccount.yaml
redis/templates/role.yaml
redis/templates/rolebinding.yaml
redis/templates/scripts-configmap.yaml
redis/templates/secret-svcbind.yaml
redis/templates/secret.yaml
redis/templates/sentinel/hpa.yaml
redis/templates/sentinel/node-services.yaml
redis/templates/sentinel/ports-configmap.yaml
redis/templates/sentinel/service.yaml
redis/templates/sentinel/statefulset.yaml
redis/templates/serviceaccount.yaml
redis/templates/servicemonitor.yaml
redis/templates/tls-secret.yaml
redis/.helmignore
redis/README.md
redis/charts/common/Chart.yaml
redis/charts/common/values.yaml
redis/charts/common/templates/_affinities.tpl
redis/charts/common/templates/_capabilities.tpl
redis/charts/common/templates/_compatibility.tpl
redis/charts/common/templates/_errors.tpl
redis/charts/common/templates/_images.tpl
redis/charts/common/templates/_ingress.tpl
redis/charts/common/templates/_labels.tpl
redis/charts/common/templates/_names.tpl
redis/charts/common/templates/_resources.tpl
redis/charts/common/templates/_secrets.tpl
redis/charts/common/templates/_storage.tpl
redis/charts/common/templates/_tplvalues.tpl
redis/charts/common/templates/_utils.tpl
redis/charts/common/templates/_warnings.tpl
redis/charts/common/templates/validations/_cassandra.tpl
redis/charts/common/templates/validations/_mariadb.tpl
redis/charts/common/templates/validations/_mongodb.tpl
redis/charts/common/templates/validations/_mysql.tpl
redis/charts/common/templates/validations/_postgresql.tpl
redis/charts/common/templates/validations/_redis.tpl
redis/charts/common/templates/validations/_validations.tpl
redis/charts/common/.helmignore
redis/charts/common/README.md
Do a quick find sed
builder@DESKTOP-QADGF36:~/Workspaces/valkey$ cd redis/
builder@DESKTOP-QADGF36:~/Workspaces/valkey/redis$ ls
Chart.lock Chart.yaml README.md charts templates values.schema.json values.yaml
builder@DESKTOP-QADGF36:~/Workspaces/valkey/redis$ find . -type f -exec sed -i 's/redis/valkey/g' {} \;
I can see that won’t work because there are dependant containers like exporter and sentinel that don’t exist yet in Docker hub
builder@DESKTOP-QADGF36:~/Workspaces/valkey/redis$ cat Chart.yaml
annotations:
category: Database
images: |
- name: kubectl
image: docker.io/bitnami/kubectl:1.29.2-debian-12-r3
- name: os-shell
image: docker.io/bitnami/os-shell:12-debian-12-r16
- name: valkey
image: docker.io/bitnami/valkey:7.2.4-debian-12-r9
- name: valkey-exporter
image: docker.io/bitnami/valkey-exporter:1.58.0-debian-12-r4
- name: valkey-sentinel
image: docker.io/bitnami/valkey-sentinel:7.2.4-debian-12-r7
licenses: Apache-2.0
apiVersion: v2
appVersion: 7.2.4
dependencies:
- name: common
repository: oci://registry-1.docker.io/bitnamicharts
tags:
- bitnami-common
version: 2.x.x
description: Redis(R) is an open source, advanced key-value store. It is often referred
to as a data structure server since keys can contain strings, hashes, lists, sets
and sorted sets.
home: https://bitnami.com
icon: https://bitnami.com/assets/stacks/valkey/img/valkey-stack-220x234.png
keywords:
- valkey
- keyvalue
- database
maintainers:
- name: VMware, Inc.
url: https://github.com/bitnami/charts
name: valkey
sources:
- https://github.com/bitnami/charts/tree/main/bitnami/valkey
version: 19.0.1
We could go down the path of building them all and using our own containers, or just wait until such time we see some official bitnami containers here
Summary
Today we built and ran Valkey, the Open-Source Redis fork, locally. We actually didn’t try building the client as we were testing how well Valkey works with the current Redis-cli. We then pivoted to building and running Valkey in Docker and lastly, we sort of shoe-horned it into Kubernetes using a bastardized Redis helm chart.
It wouldn’t take much to go the next mile to build out all fresh containers and charts. I would rather get those from the OS project or Bitnami.
