Data Platform in Power & Heat Industry

9 out of 10 E&U companies in DACH name "data availability" and "data integration" among the top challenges they encounter when developing customer-facing applications. These concern omnichannel self-care solutions in particular. Such systems allow consumers to obtain information about services,invoices and energy consumption from different business lines. They empower users to manage contracts and services themselves. And they also have great potential of becoming a truly engaging and scalable environment for delivering value-added digital services based on IoT and telemetry.

The same challenge and aspirations were shared by our client, a leading energy provider in Poland, a district heating company, subsidiary of a renowned European E&U corporation. The answer and solution provided by Hycom were Data Platform.

What is Data Platform?

Data Platform is a custom-built cloud-based solution made of proven technology stack optimized for data storage and processing. It combines two architectural concepts: Data Lake and Data Hub.

Data Lake is responsible for data ingestion and storage. Unlike traditional Data Warehouse, it is based on the "schema on read" paradigm. It means that data can be collected and stored in DL in any form and specific use cases do not have to be defined upfront. Cloud storage is scalable, relatively cheap and optimized for different types of data: structured, semi-structured and unstructured. Constantly monitored pipelines retrieve data from multiple sources, using any available integration patterns and APIs, making sure that the influx is constant and in accordance with SLA. Gaps, missing data and other anomalies are detected and reported. Data is stored in a raw form (so that it can always be referenced in its "natural" form) but also sanitized and cleaned for further use. If such a need occurs, personal data can be anonymized on the fly before they are stored on the platform.

Data Hub is a constellation of applications (microservices or serverless) specifically designed to integrate, analyze, aggregate and process any available data into datasets needed by clients. It is basically a "service" layer transforming raw and pre-processed data into valuable information that can be used by business applications. For certain business domains, a concept of canonical/common data model is used, which allows client applications to abstract and break away from constraints and limitations imposed by legacy Systems of Records.

Data Lake and Data Hub are surrounded by CI/CD, monitoring and security frameworks enabling convenient cloud infrastructure management, deployments and testing automation, as well as agile-style iterative delivery. To avoid turning Data Lake into a "Data Swamp", governance standards and processes are implemented, so that transparent data lineage can be provided and datasets themselves can be managed like true business assets.

Data Platform provides one more important capability – it is a natural environment for Data Science and complex analytics. Apart from integrating and processing data to be used by other applications, wherever there is a large data collection, there is also a significant hidden potential for business knowledge discovery. By using exploratory data analysis, statistical methods and Big Data tools and techniques, data scientists can help turn data into information and information into knowledge.

Why Data Platform is particularly suited for Energy&Utility companies?

Compared to OOTB Customer Data Platform-class products that are typically focused on gathering and analyzing generic and marketing-related customer data, tailor-made Hycom's Data Platform provides a much broader spectrum of capabilities and business domains to be covered. E&U companies in particular generate huge amounts of data from their core operations. The proliferation of IoT, smart metering, more precise and sophisticated telemetry systems make it possible to collect operational data and share them across systems - so that they can be used for various purposes. Data Platform makes it possible not only to integrate and match information about customers, contracts, services, billing accounts, invoices and payments from different business lines and Systems of Records. It also takes great advantage of large volumes and streams of data storage and processing capabilities to mix "traditional" business and customer-related information with any relevant data collection.

In the case of our client, this approach led to very interesting and tangible results:

• Data Platform became a solid data foundation for omnichannel self-care application for B2B clients and B2B2C consumers, integrating customer-related data from several Systems of Records while maintaining loose coupling and introducing independent, well-designed and business-oriented common data model.

• Data Platform collects and stores data from district heating telemetry system, making it possible to monitor performance and detect anomalies occurring in thousands of heating substations in multiple locations. This gave the company almost real-time independent
  insights into operations from a completely new perspective. Thanks to heating network optimizations, it contributed to savings and cost reductions.
  

• Data Platform gathers, integrates and stores readouts from IoT meters and sensors, allowing better customer service for end consumers (B2B2C) - by providing them with information about actual energy consumption and cost.

• Data Platform enabled the development of a new business model and new revenue streams from value-added digital services that rely strictly on data.

Our experience in the Energy & Utilities industry has enabled us to create a solution that meets both customer and user needs.