Apache Kafka for energy and utilities

The entire Dutch grid runs on Axual

Four grid problems, one streaming layer

Real-time grid monitoring

Process meter and grid data as it arrives to detect load fluctuations in real time.

Predictive maintenance

Stream asset and sensor data continuously to detect anomalies before outages occur.

Allocation and reconciliation

Combine production and consumption data for accurate imbalance calculations.

Security and compliance

Control access, validate schemas, and audit every platform change.

Diagram showing a three-layer stack with Axual at the top including connectors like Databases, APIs, SaaS, Data lakes, and Message Queues; Kafka in the middle with components such as Kafka Connect, Kafka Streams, and Brokers; and Infra at the bottom with Deployment, Kubernetes, Cloud, Operators, and Monitoring components.

Kafka Connect based source and sink connectors, managed and monitored in the same interface as topics and applications

Axual Distributor for message, schema, and offset distribution between clusters, including across on-premises and cloud

REST Proxy for systems that cannot run a Kafka client

Topic, application, and connector management in one interface, provisionable through the Axual Terraform provider

Java, Python, and .NET clients for applications that can

KSML for stream processing pipelines defined in YAML instead of Java

Multi-AZ and multi-cluster deployment, with cluster certificate expiry now visible in the UI and notified before expiry

Runs on Kafka 4 with KRaft, with a documented migration path from ZooKeeper

Controls

Authentication: mTLS, SASL SCRAM-512, and OAuth, with OAuth now supported across all data plane components including the Apicurio schema registry

Authorisation: role-based access control, ownership groups, and approval workflows on every access request

Environment separation between development, testing, and production, with separate configuration per environment

Field level data masking in Topic Browse, defined by the schema owner and extendable per topic and per environment, with a strict mode that prevents unmasking entirely

Secrets managed through HashiCorp Vault

Audit history covering creates, modifies, deletes, access requests, and approvals across platform resources, filterable by resource, actor, action, and time range

Overview graph showing every producer and consumer per topic

DORA
compliant

ISO 27001 compliant

GDPR
compliant

AICPA
SOC 2 Type II

View our ISO 27001 certification

Tennet

Transmission system operator, Netherlands and Germany 24,000+ km of high-voltage grid. ~43 million end users served. Moved from batch to real-time processing for continuous grid visibility. Data plane and control plane separated, self-service enabled with centralised security and audit.

Eneco

2M+ residential and business customers 200% faster data processing than the batch systems it replaced. 1M+ messages per day at sub-second latency. 30% energy savings in district heating operations. Multiple teams managing their own streams independently.

Enexis

Distribution system operator, 2.7M+ connections 70+ real-time streams for smart meter and grid data. 60+ applications consuming meter events. 150+ developers on a shared platform. Manual AMR data collection eliminated. Legacy systems integrated through connectors rather than custom code.

Frequently asked questions

How do energy companies use Apache Kafka?

Energy companies use Apache Kafka to move grid, meter, and asset data continuously instead of in scheduled batches. Typical uses are real-time grid monitoring, predictive maintenance on distributed assets, and allocation and reconciliation for imbalance settlement. Axual adds the governance layer on top of open Kafka: topic ownership, schema validation, access approvals, and an audit history of every change.

Can we connect OT systems like SCADA, sensors, and meters?

Yes, through Kafka Connect connectors, the REST Proxy for systems that cannot run a Kafka client, and the Java, Python, and .NET clients for systems that can. Enexis integrated legacy platforms including Tibco, Menix, and OSGI this way.

How do we handle personal data in smart meter streams?

Schema owners mark fields as masked, and topic owners can mask additional fields per environment. Masked fields are hidden in Topic Browse, and strict masking prevents them being unmasked at all. Note that this controls what users see when browsing messages in the platform, not what is written to the topic, so field level encryption or tokenisation at the producer remains the customer's responsibility.

What happens if a downstream system processes bad data?

Application owners can reset a consumer group's offsets to a chosen timestamp and reprocess from that point, without replaying everything.

How is smart meter data streamed at scale?

Smart meter data is streamed by publishing each reading once to a topic that many systems consume independently, rather than querying the source system repeatedly. Settlement, analytics, fraud detection, and customer applications all read the same stream. Enexis runs 70+ real-time streams feeding 60+ applications this way, with no manual AMR collection.

Can events move between on-premises and cloud?

Yes. Axual Distributor distributes messages, schemas, and consumer offsets between clusters, which supports on-premises grid events reaching cloud analytics.

‍

How much does Axual cost?

Pricing depends on deployment model, support level, and scale. Contact us for a quote.

The Kafka platform for European energy companies

The grid changes faster than your systems respond