Release blog 2023.2 – the summer release

AsyncAPI – an API spec for your Kafka topics

If you expose an API, whether it is a REST API or a Kafka topic, you want the users of your API to understand how they can interact with it, both from a technical perspective, as well as a functional perspective. Let’s take an example: you expose a hotel room booking API. UI developers in your organization building an interface for hotel room booking need to understand:

• Whether the API which is suggested to them is going to help them build the hotel booking interface
• The endpoint to interact with, like hostname, port and protocol

• The payload they need to send in order to create a booking for the hotel room
• The response they can expect in the  happy and not so happy flows
• Who to contact whenever they have detailed questions about the API

In a larger organization with many software teams building and exposing APIs, it is important to have this information available centrally, to reduce any delay in software design & development. For RESTful APIs the OpenAPI initiative is widely adopted as a means to standardize describing APIs. OpenAPI is extremely useful for describing APIs where synchronous interaction it taking place one-to-one,  as described in the UI ←→ API example or in the example of an API interacting with another API. However,  when a Kafka topic is your API, the interaction is asynchronous and for the one API (topic) you are exposing, you have many consumers.

The AsyncAPI Initiative is the industry standard for describing asynchronous APIs and has gained a lot of attention in recent years. It has the goal to make working with Event-Driven Architectures (EDAs) as easy as working with REST APIs.

AsyncAPI specifications contain the technical and functional details users of the API need to have to get going. It is typically a .yaml document with a couple of standard elements, such as “servers” (technical connection information)  and “channels” (e.g. topic, queue).

As of Axual Platform 2023.2 we are adding AsyncAPI support. This means that you can view and download an AsyncAPI spec for a topic in an environment. How this works can be seen in the video below.

Using the AsyncAPI spec, a developer can more easily create an application to interact with the topic.

Connector logging – what is my connector doing?

If you are integrating Apache Kafka with an external system, such as database systems, message queues or similar, Kafka Connect is the tool you need. Using Self-Service , you can easily create and configure connectors and hook them up to the topics you want to read from (SINK) or write to (SOURCE).

Up until now, you could only see the status of a particular connector in the Self-Service interface including a stack trace for when a connector crashed unexpectedly. As part of Axual Platform 2023.2, we improve the observability of your connector applications by providing Connector Logging. It has become an integral part of Self-Service and can be found on the “Logs” tab of the connector application detail page.

Eat your own dog food, that is a statement often heard in the IT industry. Well, Axual is eating her own dog food by writing the connector log entries to a Kafka topic. This brings additional benefits to you as a user of our platform. First of all, as an application owner you can easily define the retention time of the logging. The default retention is 1 week, but you can quite easily change that to whatever you want. The second and more important benefit is that you can offload the logging to a system of your choice, e.g. by creating another (SINK) connector.

Browsing and searching the connector logs is as easy as browsing data on your topics. Enter a search term (of search without), select an interval and the environment to search for logging.

Topic graph improvements

In the last two platform releases we have revamped and improved the topic graph. In this release we improved its usability even more. When you search for an entity in the topic graph, topics and applications are returned. Clicking a topic or an application will now highlight the selected topic or application and show its details and connected applications or topics. This helps you to find out immediately whether that’s the topic/application you are looking for.

Terraform provider: topic approval request support

We have recently released the latest version of the Terraform Provider for Axual which introduces support for:

• Managing public environments: you are now able to  manage public environments from terraform files
• Creating, approving/denying and deleting application access requests: create an authorization request for your application to produce/consume from a topic.

The latest version of the Terraform Provider for Axual is 1.1.0 and it is available in the Terraform Registry. Be sure to check out the examples, especially the application_access_grant and application_access_grant_approval documentation pages, which should help you understand how to create and approve requests. Did you find any issues while using the provider? Please let us know.

Start your Kafka journey with Axual

Got excited from reading this post? Are you struggling with Kafka in an enterprise setting? We are here to help. Here are a couple of next steps you could take:

– Start a free trial

– Request a demo and talk to our experts

– View other content and customer case studies

‍