Release blog 2023.1 – the spring release

We are well into spring again and the flowers start popping up around us. What a wonderful moment to bring another platform release to the public. We are happy to announce Axual Platform 2023.1.

Viewing topics and applications in various scopes

If you are in an organization that is successful in leveraging Apache Kafka, dozens or perhaps hundreds of topics are created and an equally large number of applications are interacting with those topics. Keeping a good overview of what is happening on the platform is definitely a huge challenge. In a previous version of the platform we introduced a way to browse through environments to see which topics and applications are active there and how they are interacting with each other. In this release we are adding the possibility to select different scopes to display the topics and applications in.

You can select the following scopes:

• All teams: the entire organization
• My teams: all teams that the logged-in user is a member of

Consumer lag – how is my consumer doing?

As an application developer, especially in a DevOps setting, you need to make sure your application is running as it should. There are various metrics that you can look at and when it comes to applications interacting with Kafka, there is a specific one that is very extremely important.

Intermezzo: partitions and parallelism on Kafka
When you create a topic on Kafka, you define the number of partitions that you want to create for your topic. Those partitions are important to enable parallelism on Kafka as:

1. Brokers are the leader for 1..n topic partitions
2. All read-write operations from clients (producer/consumers) happen through the leader of the topic partitions.

For example, if you have a topic with 3 partitions and a 3 broker cluster, and the partitions are evenly distributed on the cluster, each broker takes up a third of the messages on the topic.

A core characteristic of Kafka is that messages on 1 topic partition can only be read by 1 instance of a consuming application at the same time. In a situation where you have a high input on the topic, e.g. 300 messages per second, and your consumer instances together cannot process more than 250 messages per second, you will start to lag. This is called consumer lag. In release 2022.3 we introduced the Consumer Lag metric to the metrics API.  As part of release 2023.1 we are showing this metric in the user interface. If you have a consuming application, the consumer lag in records will be shown on the connection between the topic and the application.

Because you might consume from multiple topics, the lag is shown for each topic you are consuming from.

If you want to get the consume lag programmatically, please check the API documentation.

Huge improvement of the online documentation

In the last couple of months we have made significant improvements to our online documentation. We have restructured the content of the documentation and we have also made it possible to search through the entire collection of documentation repositories.

Please check it out by going to https://docs.axual.io and let us know what you think.

In other news: Launching Axual Governance

Only a week ago we released Axual Governance. With Axual Governance, we are bringing data governance and self-service capabilities to organizations that are already using Kafka. If you have an Aiven or Confluent Cloud Kafka cluster, you can start the onboarding right away. For Plain Kafka and Strimzi users, we are going to add support for onboarding soon.

Read more about Axual Governance in this blog or sign up  for yourself!

Strimzi upgrade

Axual Platform is based on the Strimzi Operator. For the 2023.1 release we have upgraded to the latest Strimzi release,  Strimzi 0.34.0 . It allows for an upgrade to the 3.4.0 release of Apache Kafka. The operator instructions contain upgrade steps related to Axual Operator 0.10.0 to upgrade to Kafka 3.4.0. You can find the operations documentation here.

Axual Operator 0.10.0 is effectively the last release of Axual Operator. As the Strimzi community introduced features to the operator which allows it to be configured and plug in functionality, like a PrincipalBuilder, there is no need anymore for us to fork Strimzi Operator.

On-premise installations of our platform can in the future use Strimzi operator with a reference to  images prepared by Axual. We will send out an update about this in the coming months.

A more extensive list of  updates done to Axual Platform can be found in the release notes.

‍