In one of my last posts I presented SMM as a monitoring and management tool for Kafka. Today I want to show another alternative for managing Kafka.
This product is KaDeck, from a German company called XeoTek.
Installing and running
They offer KaDeck in two flavors: A local desktop installation and a web version. Lately they also released a demo version that runs an entire Kafka cluster along with KaDeck in a docker container and can run on a laptop and demonstrates KaDeck.
To Install the demo version just run this on a computer with docker installed:
docker run -it -p 80:80 xeotek/kadeck-allinone:latest
When it’s done starting, point your browser to http://localhost and login with admin:admin and start exploring it.
If you want the real web version then you need to register in their website and create a team first. Yes, KaDeck is aimed at teams of people working together. You create a team online and shortly you receive a mail with team id and a secret key that you will use when running KaDeck web..
It seems that there is no host installation of KaDeck right now (rpm package, binaries or so) but only a Docker image. This forces the user to run it in Docker or a Kubernetes cluster (or one of it’s commercial distributions such as OpenShift). Another thing is that checks your license online each time you start the container. If you want to use t offline you have to do a process of offline activation. You can reach it from the administration menu.
Then you use the secret and team id to run KaDeck in a docker container:
docker run -d --network host --name=kadeckweb -e xeotek_kadeck_secret=[the secret code] -e xeotek_kadeck_teamid=[team id] -e xeotek_kadeck_port=80 --rm xeotek/kadeckweb
Point your browser to your host on port 80 and login with admin:admin.
Overview
Then you will have an opportunity to add a cluster by supplying the name and port of one or more of your Kafka servers:
The main toolbar s located on the left of the screen and has four main functions: Overview, topic browser, topic inspector and audit. It’s a little confusing to have both topic inspector and topic browser.
The overview screen shows, well…, an overview of the cluster, number of brokers, topics and , along with configuration parameters of the cluster in the lower pane:
I will not go through all of the options and sub-options, and there are plenty of them. We will do just a quick overview and highlight some special features that separates KaDeck from other tools I know. You are welcome to run the demo version (or the full version) and explore it yourself.
Topic browser
The topic browser deals with the data itself, the messages inside Kafka. It is loaded with options and features:
On the left there is a tree view showing the topic and the views under them (I’ll explain about views in a minute).
The main pane shows the flow view which demonstrates the message distribution among partitions, but it also have tabs for filters, time distribution (how many messages per time unit) and others.
On the bottom pane you can see the actual data in the topic. Right now it is static data and you have to hit the refresh button to see new messages. Upcoming versions of KaDeck will support a very cool feature of real time, self updating view of the messages that are going into the topic.
If you click on one of the rows you will get a detailed look of this message.
You can filter records by creating a filter that uses a <=> operators criteria or a javascript filter using the quick processor tab.
Another unique option that I liked is the ability to create views. Views are actually filters that are saved for later use. This way you can have your most frequently used filters ready for use in no time. They are then shown under their parent topic. Here is how to create and use them.
Topic inspector
While topic browser deals mostly with the data itself, topic inspector takes care of the topic itself:
The topic list is on the left while on the right you can see parameters like consumer lag and consumer status which are important metrics to monitor in a Kafka cluster.
If you choose the topic configuration tab you will see a list of configuration parameters that you can edit directly from KaDeck:
Audit log
There is also the audit log screen that logs all the actions done in KaDeck but it cannot fetch actions on the cluster that were done outside KaDeck:
There is also an administration menu at the upper right corner where you can create new users and roles and grant access to them (You don’t want all users to connect as admin) and manage licensing.
The first user is free and additional users cost 15$/month (If I understand it right). 24/7 support will add some cost too.
Conclusion
There are many free Kafka tools out there (I usually use Cloudera SMM in my daily work, and there are others too). In the beginning KaDeck seemed to me like just another Kafka management tool, but as I experimented more and got deeper into it I started to appreciate it.
It has all the functionality needed to manage Kafka cluster (that can be found in other tools) along with some unique options that I did not see in other tools like views, javascript based filters, real time messages display (which is still in beta) and more. I also liked the user interface which is compact and simple but has all the needed functionality. Alerting capability would be a good addition though. It is a decent choice if you are willing to pay for some extra functionality.
* Sharped-eye readers may notice that some of the screenshots are from the demo distribution while others are from my own cluster. I did used a mix because my test cluster had poor traffic and only two topics, so the images weren’t so impressive like in the demo distribution. So for some screenshots i just used the demo.