Kafka is a very powerful and robust messaging system that is widely used in big data systems. However, it is not that good at its own housekeeping. Each topic in Kafka is divided into partitions that enable parallel publishing and consuming of data and each partition has several replicas (usually Three) which are spread across… Read More »
In one of my last posts I presented SMM as a monitoring and management tool for Kafka. Today I want to show another alternative for managing Kafka. This product is KaDeck, from a German company called XeoTek. Installing and running They offer KaDeck in two flavors: A local desktop installation and a web version. Lately they… Read More »
Streams messaging manager (SMM) is not new to HortonWorks users, but since I was mostly using Cloudera I never had the opportunity to use it. Following the merger of Cloudera and Hortonworks in early 2019, many good products that were originally part of HDP finally made their way into the Cloudera platform including SMM. Cloudera’s… Read More »
Handling streaming data in Kafka isn’t an easy task. Kafka itself does not offer mechanisms for data transformation or combining data a from different streams. You will have to write your own programs to do it. Confluent has recognized this problem and developed an open source tool called KSQL. It reads Kafka topics and exposes… Read More »