Event Streaming as the next step in Eneco’s energy transition
Sustainability is an important topic in the energy sector. Eneco’s mission is “Sustainable energy for everyone” and wants to help its customers take the next step in their energy transition. This requires knowledge in two areas, specifically: knowledge of the market and of the customer. This knowledge is obtained from data. Not only does more data arise every year, but this data must also be available and processed as fast as possible. Because “real-time data” plays an increasingly important role in the energy transition, Eneco decided in 2019 to set up an Event Streaming Platform. This project was awarded to the partnership of Conclusion Mission Critical and Axual.
“Real-time data is the future,” says Ronald Root, Manager Data Science and Business Innovation Eneco. “With real-time data, we can immediately help our customers with their questions and give advice on how to use energy even more sustainably. Take, for example, customer questions such as “Why is my energy consumption so high at the moment?” With real-time data we can immediately see this and find out, together with the customer, what is going on in their house at that moment. “
Joleen Breij, Head of Data Eneco, is convinced that Eneco must become even more of a data-driven company to realize its mission. The Event Streaming Platform is one of the technical foundations that underlie this.
With the Event Streaming Platform, Eneco can process its data in real-time twice as fast as before, with millions of messages elaborated per minute as a result. In addition, the platform enables real-time access, so that, in collaboration with IoT (Internet of Things), (real-time) analytics and control, it is possible to respond better to the demand and supply of heat supply for customers. This can lead to energy savings of 20-30% which is the next step to make the Netherlands more sustainable.
To achieve its objective, Eneco launched an “Event Streaming Platform” in the market in 2019.
An Event Streaming Platform is a “data access environment” through which Eneco can obtain data from all kinds of different sources in a structured manner. These are events where telemetry data describes exactly what is currently going on with a particular entity. Think of an anemometer, electricity meter or, for example, a light sensor.
“By having access to this real-time data, Eneco can therefore use its data flexibly to help the organization and its customers even better.” Said Iem Smid, Project Manager Program and Portfolio Management Eneco. As a result, customers will soon receive personalized advice about, for example, the status of their insulation or real-time abnormalities in energy consumption. This results in being able to use energy more sustainably.
By means of a Request for Proposal, Eneco has set out the assignment for building, implementing and managing the new Event Streaming Platform with Apache Kafka as a basis. Based on assessment criteria on a functional and operational level, the partnership of Axual and Conclusion Mission Critical scored the best. In addition, Eneco had confidence in both organizations because Axual had a proven platform built around Kafka and because Conclusion Mission Critical already had a proven track record in other Eneco projects.
“Conclusion Mission Critical has shown us several times that it will achieve good results in this type of field. We had every confidence that they could do this project. ” – Joleen Breij
For the platform itself, Conclusion Mission Critical went looking for a partner with technical expertise, in order to be able to offer Eneco the best possible solution. Eneco was already in contact with Axual in this regard and this is how the collaboration between both parties took place.
“We could have chosen to design and build a platform ourselves, but then we would have had to invent a lot of wheels ourselves to ensure an overview of all data flows and access was done right. With Axual these concerns are taken care of. ” – Joleen Breij
With Axual as Kafka specialist and Conclusion Mission Critical as ultimately responsible (management) for the Event Streaming Platform, Eneco had found the ideal partners for its Event Streaming Platform.
Short time-to-market due to close cooperation
In just 2 months, Axual delivered its platform within Eneco’s Microsoft Azure environment. The platform contains a self-service environment that enables Eneco teams to define and manage new data flows. This allows it to realize new use cases quickly and without the intervention of third parties.
“Setting up the platform with Axual and the collaboration with Conclusion Mission Critical went very well. Eneco has always experienced this as very positive. ” – Ronald Root.
In order to keep the Event Streaming Platform available at all times, to protect it against cybercrime and to keep it agile for any adjustments, Conclusion Mission Critical is responsible for the management of this platform. This entails technical application management, including vertical chain management, 24-7 monitoring, monitoring and resolution of incidents, and the implementation of changes. All this to guarantee 24-7 availability of the platform and unburden Eneco so that it can focus on its core business.
The management of the platform took a little more effort than setting up the platform. With good cooperation and open communication between the parties involved, Conclusion Mission Critical has been able to properly organize the management.
The result: 100% faster data processing & 20-30% energy savings
With the Event Streaming Platform, Eneco is now able to monitor and process all data streams twice as fast. At the moment, Eneco’s most customer-oriented applications and websites are connected to the platform so that Eneco can respond in real-time to questions that come in from, among others, the customer. Another example of real-time processing of data is monitoring power and heat consumption.
“With the help of data, it can be better determined how much heat should be put into the pipes. In this way we can save 20-30% energy and this is immediately beneficial for the customer. ” says Ronald Root. With this, Eneco, together with its customers, is taking a new step in the energy transition / sustainable energy use.
What will the future bring?
The ESP platform runs stable and does exactly what it should do according to Joleen Breij. “This helps the organization a lot in connecting new data flows for the platform”.
There is already a lot of interest from within the organization for new use cases on the ESP platform. “Take IoT and asset management, for example. With asset management we try to ensure that all meters in the country are adjusted in such a way that they can access real-time data via the ESP platform towards the right services and customer service. So that they can help the customer immediately. ” Said Rob Brouwers, Data Integration Expert Eneco.
Other applications that are currently being looked at are connecting the HR processes to the ESP platform and improving customer interaction; by keeping all data up-to-date via the ESP platform, so that it is adapted across the board and thus the same customer image remains across the various applications.
Eneco is expected to have connected more than twenty data streams to the ESP platform by the end of 2020. “A wonderful result for this project!” said Ronald Root.
“Conclusion Mission Critical and Axual are really amazing partners to work with in terms of proactivity, collaboration, thinking along and trust. They are professional parties with whom we get a good connection. People who really understand and think a step further with us. Definitely a combination that I would recommend. ” – Ronald Root.