Product 23 Apr 2021

Why Axual Uses Strimzi Operator?

However, running Kafka on Kubernetes isn’t without its problem. One of the problems is configuring and managing Kafka components on Kubernetes. For Axual, the solution comes in the form of Strimzi Operators.

Running Kafka on Kubernetes seems to be a natural choice, given both technologies’ serverless and scalable traits. Likewise, we’ve been offering a real-time data streaming solution that’s built around a Kafka-on-Kubernetes foundation.

Kafka is a distributed messaging system that facilitates real-time data processing. It’s fault-tolerant, highly-scalable, and low-latency. However, it needs an environment to run where its benefits are not compromised.

Kubernetes, being a cloud-native scalable platform for managing container-based services, is the perfect match for Kafka. Both are cloud-development friendly and meant to scale according to operational needs.

However, running Kafka on Kubernetes isn’t without its problem. One of the problems is configuring and managing Kafka components on Kubernetes. For Axual, the solution comes in the form of Strimzi Operators.

What is Strimzi Operator?

Strimzi operator is a tool made available under the Strimzi open source project, meant to simplify how Kafka is configured, deployed, and managed on Kubernetes. It allows developers to use familiar Kubernetes processes to setup Kafka without digging deep into the infrastructure’s technicalities.

With Strimzi, configuring Kafka in the Kubernetes environment is as simple as writing coded instructions, which are then executed by the underlying platform. Strimzi isn’t only about providing a user-friendly Kafka configuration environment but also built with fundamental security features.

Different Types of Strimzi Operator

There are 3 types of Strimzi operators that’ll let you set up Apache Kafka on Kubernetes within minutes.

They are:

To use the operators, you’ll need to set up the associated resources. You can do so by creating instances of Custom Resource Definitions or CRDs as an extension to Kubernetes resources. The operator execution is based on the configurations on the respective CRDs.

How Strimzi Operator Simplifies Kafka Deployment On Kubernetes

Kubernetes is a good container platform for running stateless applications or services. It is, however, not a natural fit for stateful applications like Kafka. If you attempt to set up Kafka on Kubernetes, there are many questions that need careful considerations.

For example, you need to place Kafka brokers on different Kubernetes nodes and ensure each node does not have more than one replicas. That’s where Strimzi’s rack awareness is handy. It spreads Kafka broker pods through all the available zones to prevent disruptions when one of them goes down.

Before you can deploy the Kafka cluster for production, you’ll need to conduct a health check on the Kubernetes pods. The liveness probe will automatically restart a pod if it fails to respond. Meanwhile, the readiness probe is used to determine if the Kafka pod can start processing incoming requests.

With Strimzi, you don’t need to go through the complicated process of setting up the probes. All you need to do is to run the bash script for the respective probes to get the results.

Strimzi operators remove many obstacles of running Kafka on Kubernetes. It’s the foundation of Axual Platform, which has proved reliable for many of our clients.

How Axual Uses Strimzi

For Axual platform, we use Strimzi Cluster Operator for deploying Kafka in Kubernetes. We do not use Topic and User Operator as we have our own Self Service tool that allows developers to manage their topics and ACLs with enterprise level data governance and security.

As versatile as Strimzi is, we could not use integrate it directly with Axual platform. One of the key design principles behind the platform is multi-tenancy. At Kafka level this is done with the help of a custom principal builder that stores not just the DN of the certificate but also the DN of CA (intermediate and Root) that signed it. As long as unique Root CAs are used per tenant, it is guaranteed that no two tenants can use the same certificate for authorizing access to a topic.

Unfortunately, Strimzi does not have any multi-tenancy features. It also blocks the “principal.builder.class” config. This meant we could not use our custom principal builder for multi-tenancy without modifying the Strimzi code. So we decided to fork Strimzi Cluster Operator and add support for our principal builder. With the forked Strimzi operator we were able to support multi-tenancy with mutual TLS.

Other blogs

2 months ago

Apache Kafka drives Rabobank Real-Time Financial Alerts

Rabobank used Kafka Streams APIs to do real-time alerting on financial events for their customers. Learn more about the use case in this blog.

Abhinav Sonkar
Technology 2 months ago

Why and How You Should Deploy Kafka Clusters on Kubernetes

Abhinav Sonkar
Product 3 months ago

Axual Release Update 2021.2

The summer release 2021.2 is here, it contains many quality and stability improvements and some interesting new features. Read the release blog to find out more.

Abhinav Sonkar

Apache Kafka is great, but what do you do
when great is not good enough?
See what Axual offers on top of Kafka.

Start your free trial
No credit card required