Release blog 2024.3 - the Autumn release
Let’s dive into the highlights of the 2024.3 release and see how we’re equipping you to confidently handle the next season of data challenges.
On this page
Try KSML in self-service and help us improve with your feedback
As the leaves turn golden and the crisp autumn air fills our days, we’re excited to present the Autumn Release of Axual Platform 2024.3.
Autumn is a time for reflection and preparation, and we’ve applied that mindset to this release, ensuring that the platform meets your needs while adapting to the ever-changing data landscape. From new schema enhancements to improved self-service tools, we’ve incorporated user feedback and focused on making this release more intuitive, efficient, and future-ready.
Whether you leverage the power of KSML in self-service, work with JSON Schema for more formalized data contracts, or take advantage of new topic tagging and search functionalities, this release is designed to give you more control, insights, and better performance.
Let’s dive into the highlights of the 2024.3 release and see how we’re equipping you to confidently handle the next season of data challenges.
KSML in Self-Service
Try KSML in self-service and help us improve with your feedback.
What is the feature?
The latest version of KSML (1.0.2) is now fully integrated into the self-service portal, allowing users to build and manage stream-processing applications directly within the Axual platform.
What is its purpose?
KSML simplifies stream processing by enabling developers to write and deploy stream applications using YAML syntax. This integration into self-service ensures users can configure, monitor, and adjust their streaming logic without external dependencies.
Who is this useful for?
This feature is handy for developers and data engineers building real-time processing applications but prefers to manage their workflows in a self-service, low-code environment.
Explanation in more detail
With KSML now in self-service, users can:
- Deploy stream processing logic with configurations directly through the Axual platform.
- Monitor real-time stream data, apply transformations, and ensure smooth, scalable processing.
Check out the KSML 1.0.2 release notes for a complete list of updates and improvements.
Here’s a glimpse of how KSML runs within the self-service portal
Creating a new KSML application
Configuring the stream processing topology in KSML application
Viewing application logs
JSON Schema Support
What is the feature?
Support for JSON Schema as a new schema type, enabling users to define structured data contracts for JSON messages.
What is its purpose?
This feature formalizes how JSON data is transmitted between applications by enforcing a schema-based approach. It ensures data consistency and improves message validation, which is essential in complex, large-scale data systems.
Who is this useful for?
This feature benefits users working with JSON-based data pipelines, mainly developers and architects, who must establish clear data contracts between teams or services.
Explanation in more detail
With JSON Schema support, users can now:
- Define and register JSON schemas alongside Avro and more.
- Enforce structure and type consistency in JSON data flows.
- Leverage formalized JSON contracts for better inter-service communication.
Here’s a glimpse of how it works.
Uploading JSON Schema
Topic Tags and Search on Tag
What is the feature?
Introducing the ability to assign tags to topics and perform searches based on these tags.
What is its purpose?
This feature enhances the platform's discoverability and organization of topics by allowing users to tag them with custom labels and perform tag-based searches.
Who is this useful for?
This feature is useful for teams managing a large number of topics. It benefits application owners, data engineers, and platform administrators who need an easier way to categorize and locate specific topics.
Explanation in more detail
Users can now:
- Assign custom tags to topics for easy identification
- Use the search functionality to filter topics by tag, improving overall navigation and management.
Topics with tags assigned
Searching topics by tags
Schema Ownership Management
What is the feature?
Enhanced schema ownership management, including a new transfer ownership feature.
What is its purpose?
This update allows users to easily transfer schema ownership to other users, ensuring proper accountability and management when team structures or responsibilities change.
Who is this useful for?
This feature is handy for schema owners and platform administrators who oversee schema governance and need to ensure clear ownership during handovers or team transitions.
Explanation in more detail
The transfer-ownership feature allows users to:
- Seamlessly transfer schema ownership within the platform to another user.
- Ensure continuity in schema management and oversight.
Distributor Update (5.3.2)
What is the feature?
Updated Distributor (version 5.3.2) with separate Docker images for Connect and Strimzi versions.
What is its purpose?
This update ensures compatibility between different versions of Connect and Strimzi, allowing users to select the image that fits their deployment environment.
Who is this useful for?
Platform administrators and DevOps teams will find this update useful when managing distributed environments. It provides more flexibility in version control and reduces conflicts during updates.
Explanation in more detail
With separate images for different versions, users can:
- Select the appropriate version of the Distributor for their Connect/Strimzi setup.
- Simplify updates and rollbacks when managing distributed environments.
General Improvements
Delete Confirmation Enhancements
- When deleting a topic, application, or environment, users will see all affected resources before confirming the deletion, helping prevent accidental loss of critical resources.
In our release notes, you will find other, more minor, updates to our product, which we are continuously improving with your feedback. Check them out here.
Begin your Kafka journey with Axual
Inspired by what you've read? Facing challenges with Kafka in an enterprise environment? We're here to assist. Here are your next steps:
Answers to your questions about Axual’s All-in-one Kafka Platform
Are you curious about our All-in-one Kafka platform? Dive into our FAQs
for all the details you need, and find the answers to your burning questions.
KSML is a wrapper around Kafka Streams that allows for development of low code stream processing applications.
KSML provides a new declarative approach to unlock Kafka Streams to a wider audience. Using only a few simple basic rules and Python snippets, you can write streaming applications in very little time.
KSML stands for Kafka Streams Modeling Language. It is a domain-specific language (DSL) used to simplify the development of streaming applications.
Related blogs
Apache Kafka has become a central component of modern data architectures, enabling real-time data streaming and integration across distributed systems. Within Kafka’s ecosystem, Kafka Connect plays a crucial role as a powerful framework designed for seamlessly moving data between Kafka and external systems. Kafka Connect provides a standardized, scalable approach to data integration, removing the need for complex custom scripts or applications. For architects, product owners, and senior engineers, Kafka Connect is essential to understand because it simplifies data pipelines and supports low-latency, fault-tolerant data flow across platforms. But what exactly is Kafka Connect, and how can it benefit your architecture?
Apache Kafka is a powerful platform for handling real-time data streaming, often used in systems that follow the Publish-Subscribe (Pub-Sub) model. In Pub-Sub, producers send messages (data) that consumers receive, enabling asynchronous communication between services. Kafka’s Pub-Sub model is designed for high throughput, reliability, and scalability, making it a preferred choice for applications needing to process massive volumes of data efficiently. Central to this functionality are topics and partitions—essential elements that organize and distribute messages across Kafka. But what exactly are topics and partitions, and why are they so important?
Strimzi Kafka offers an efficient solution for deploying and managing Apache Kafka on Kubernetes, making it easier to handle Kafka clusters within a Kubernetes environment. In this article, we'll guide you through opening a shell on a Kafka broker pod in Kubernetes and listing all the topics in your Kafka cluster using an SSL-based connection.