We brought together a panel of experts to discuss the role that regulation plays in supporting DNOs and unlocking sector-wide innovation, in our February webinar hosted by Icebreaker One Co-Founder and Programme Manager, Gea Mikic:

• Charlotte Hillenbrand, Product Strategist, Icebreaker One
• Sara Vaughan, Non Executive Director, Elexon
• Liam Bennett, Senior Manager – Data Policy and Regulation, Ofgem

The panel started by discussing what’s changed for DNOs and the wider energy market over recent decades, before exploring how the sector can work together to achieve net zero goals. Here’s a summary of the key themes.

1. The impact of transition

Increases in distributed generation require Distribution Network Operators (DNOs) to take on system operator functions; shifting their role from passive/reactive energy distribution to such as and active network management, using new technology and real-time data to make interventions on the network: to the transition from Distribution Network Operator to Distribution System Operator (DSO).

“The distribution network used to be seen as passive. It was basically a conduit for getting power from one place to another. Today is very different. In the last decade, the usage patterns of Britain’s local electricity distribution networks have changed rapidly. There’s been a sharp increase in more low carbon generation connecting to local networks, as both consumers and businesses invest in their own generation equipment” says Sara Vaughan, Non Executive Director, Elexon.

Sara explains: “In a DSO model, the very last thing that the network will be is passive. It is sitting at the centre of thousands of potential interactions, playing host to a variety of different distributed energy resources and enabling flexibility transactions as an alternative to just investing in more network capacity. The key to this, of course, is data and digitalisation. The networks need to know where the assets are, the investors and developers need to know where the opportunities are, and consumers need to know that this is all possible. Greater data visibility and open data are essential steps towards the digital transformation of the energy sector and the acceleration to net zero”.

Are we heading in the right direction?

It’s clear that this is a pivotal moment for the energy sector. How can policy and regulation provide the structural support for this shift?

“What every regulated body needs is a degree of certainty in terms of future policy. My concern, when you couple the current world situation with the cost of living crisis that we’re seeing more generally at the moment, is around political will to keep pushing forward. We know that there are some members of Parliament already who are talking about slowing down the drive to net zero. But we will never get to net zero unless we drive for it. Once we are on that path, then we reduce our dependence on gas and its volatility. So we have to keep that decarbonisation focus and investment going.”

Sara Vaughan, Elexon

We need to work collectively to keep modernising and diversifying our digitised energy systems. Liam Bennett, Senior Manager – Data Policy and Regulation at Ofgem, emphasises: “Digitalisation is the key; driving open data up through the DNOs and up through other systems to provide value. If we keep progressing in silos and we all think that in our individual parts of the system we’re building the right thing, but none of it interacts; we don’t really get that true value in the whole system and we end up either overbuilding or we just fundamentally don’t get there.

“Each DNO having its own data sharing platform is definitely a good starting position. To make data available to consumers, to stakeholders, to Ofgem, is a really positive step. My push to DNOs would be: how do we use those open data platforms? We need to take a common approach, whether it be through an open network project or a third party; making sure that stakeholders and flexibility providers can go and access each of those different data platforms and know what data they’re going to get, in the right fashion, in the right timeliness”.

2. Focus on interoperability

Interoperability is essential if the energy industry is to achieve its 2030 climate targets. Liam states: “The wider system probably needs three key points to deliver in the next 10 years. The first is common data and metadata standards. Ofgem recently published a letter on the common information model for the Long Term Development Statement for DNOs, which is a strong step for us, and we think future standards will also be valuable. Secondly, minimum standards for data quality. Everyone in the digital energy sector needs to know that the data they receive is accessible and acceptable. Thirdly, you need clearly articulated roles and responsibilities in the sector. You need to reduce risk; we need increased regulatory certainty. And we need to understand who you interact with to get what you need to operate in that future system. The operation requires interfaces between different parts.

“We’d like to see the approach DNOs are taking to engage external providers such as Open Energy to find common interfaces. We’ve been on quite a journey so far, from the Energy Data Task Force into the Energy Digitalisation Task Force. We’ve taken quite an open approach to our regulation, collaborating with the sector and with DNOS as well. We want to take that forward. Collaborative and iterative design is the only way we’ll get the energy system we all envisage. So yes, our door is open. We want to design the system with you. Come and help us.”

Liam Bennett, Ofgem

3. Getting started

With so much to navigate, it’s important just to make a start. Charlotte Hillenbrand, Product Strategist at Icebreaker One, explains: “the challenge is always: where do we start? I’ve encountered lots of clients where that conversation has been rolling for a couple of years. And the problems are somewhat known, but the starting point still evades the teams. So that’s where Open Energy is really trying to help DNOs and other players in the sector: to get started. With Open Energy you can dive in and start accessing data; start finding open and shared datasets and see who else in the community is active.

“We’ve really tried as much as we can to facilitate DNOs to operationalise the data best practice that’s laid out in their RIIO-ED2 business plans, because we recognise that this is a really complex space. We’re focussed on removing as much friction from the process as we can, in order to get to value as quickly as possible.”

Charlotte Hillenbrand, Icebreaker One

Charlotte: “We’re indexing and connecting data, not collecting data. So the data will always remain on individual systems. What we’re facilitating is the connection and transaction of datasets in a safe environment. And in that way, you can build the picture, ingest the data you need, and start to map it”.

Get involved

• Become a member of Open Energy to use our pilot service
• Join Icebreaker One, which develops and operates Open Energy, as a supporter
• Sign up to the Open Energy newsletter

A panel of experts gathered to discuss the emerging pressures and priorities for DNOs and the wider energy system, in Icebreaker One’s January webinar hosted by Open Engagement Manager, Vichi Chandra.

“We can’t talk about anything happening in the energy market without thinking about the current context of rising gas prices, the current cost of living crisis, and what that means to people” says Dhara Vyas, Deputy Director, Energy UK. “High prices are – rightly – driving the media narrative right now and that has meant a significant shift away from the UK leading the way on decarbonisation and net zero, to the current cost of living crisis which impacts on everyone. 

“However, I think there is a significant, real concern that this should not mean we put the brakes on our journey to achieve a net zero system. It’s really important that we think about this in the context of the consumer.”

Matt Webb, Head of Enterprise Data Management, UK Power Networks highlighted the role of DNOs in making this happen: 

“We need to accelerate the connection of low carbon technologies and maximise their use in order to contribute to net zero ambitions. At the same time, we must maintain our core role of ensuring the continuity and quality of energy supply. We cannot lose sight of the fact that we need to keep the lights on”.

Matt Webb, UK Power Networks

A period of transformation

January’s Energy Digitalisation Taskforce report made a number of key recommendations detailing what is required to deliver a digitalised and decarbonised energy system.

“One that stood out to me was the call to embed a culture of digitalisation and I think that’s really important”, said Dhara.

“It’s essential that companies across the market are planning for a digital system when it comes to building the right skills and investing in digital assets and activities. DNOs are of course a vital part of that infrastructure, and the networks in fact have a much better understanding of the state of the system than perhaps a lot of other parts of the sector; so it’s vital that they share that across the market in order to help us build the system of the future”

Dhara Vyas, Energy UK

It’s a viewpoint shared by Matt. “Digital transformation is as much about culture, behaviours and ways of working. And that’s where we as a sector need to shift to be more open and collaborative. We talk about interoperability a great deal; and that’s about how we align and standardise and facilitate communication and interaction between the different players in this ecosystem. 

“The RIIO ED2 business plans that we as DNOs have published in recent months, go into significant detail about how we plan to invest and develop our digital capabilities over the coming years, right the way through until 2028. We find ourselves at a really pivotal and important point in terms of how we meet this ongoing challenge, and ultimately deliver the interoperable energy system of the future that we are all aiming for”.

The shift that is required of DNOs, and the wider energy system as a whole, is significant. As Matt explains, “in normal terms, what a DNO is and does, is quite monolithic. And all of a sudden, to find yourself in a central, facilitating role is a big change to make. Traditional network operation entails fairly passive and predictable systems, and we’re now moving into – and are in the midst of – a far more dynamic environment. We need to help facilitate new forms of service, service providers and business models; increasing competition and consumer choice as well as supporting a flexibility first approach”.

How Open Energy can help

Open Energy is a service that makes it easy to search, access and securely share energy data. It covers the full spectrum of data; all the way from open data to really hard to access, commercially sensitive shared data, where access control is important. The Open Energy service currently includes search and access control, co-designed over the last 18+ months by industry, for industry, with support from and engagement with BEIS, InnovateUK and Ofgem (who sit as an observer on the Open Energy Steering Board).

Gavin Starks, Founder and CEO of Icebreaker One, states:

“The approach that we’ve taken with IB1 and the development of Open Energy has been to ask what are the market design principles for data sharing that can scale to whole markets, and across sectors? This has led the teams to understand how to reduce the friction, and connect data between organisations, without insisting that anyone has to put their data in a particular place. Secure data exchanges are made possible by the adoption of open standards and a shared data infrastructure that underpins cohesion and interoperability across the market.

Gavin Starks, Icebreaker One

“What we’ve been focussing on is the really valuable information that tends not to be Open – and in many cases can’t be Open. As soon as you need a set of rules around that, there needs to be a set of legal contracts and a set of processes whereby the control around who can access what recommendations, when, and so on, can be codified. There’s a lot of work there on how we can give that Access Control to the people who hold the data. 

“Whether or not that’s a business-to-business context or whether it’s a consumer context; the mode of consent, the process of consent management, and rights assignment to the data are all miles away from what organisations are used to thinking about and dealing with. So there’s a massive amount of work to be done in helping to understand the problems that everybody’s trying to solve, both in terms of infrastructure, but also in understanding what the rules are and how we can bring people together so that we end up with an open marketplace”.

What are the opportunities here for DNOs?

Matt says: “When it comes to data, commerciality is certainly one of those factors we need to think about, and we need to be cognisant of the fact that we work in an increasingly competitive market. However, we need to recognize that we are part of an ecosystem now where that commercial data is key. The key to that is us working collaboratively and trying to establish the common processes and practices that make sure that we’re making consistent decisions.” 

Dhara agrees that a whole-system approach is needed: “we have identified all the pieces of the puzzle but now we need to put them together”.

“It’s not just about DNOs but everyone in the sector; the challenge is manifold” adds Gavin. “Leadership needs to come from various different points of the system – government and market leaders – and a lot of the innovation will come naturally from large organisations and smaller start-ups. The question is: how can we do all that together? 

“What we’ve tried to create here with the Open Energy programme is space for that to happen. So ultimately we’re here to help bring people together, have the right conversations and really focus on where that business value is and how that interlinks with the overall data strategy; both for individual organisations and for the sector as a whole”.

Get involved

• Become a member of Open Energy to use our pilot service
• Join Icebreaker One, which develops and operates Open Energy, as a supporter
• Sign up to the Open Energy newsletter

As the DDSG chair has commented previously, “as we look to a low-carbon economy, the data that we need to reach Net Zero at the pace we’re working to isn’t available…. the gas and electricity networks are deploying digital solutions to ensure we can achieve not only our wider decarbonisation goals, but ensure we continue and improve the level of service for our customers”.

The slides used during the discussion are below.

Please see here for our open calls to action

About the session

Simon Evans, Arup’s  Digital Energy Leader and Lee Wilson, Scottish Power’s Data & Analytics led today’s session on the Metadata Standards & Glossaries Advisory Group, with over 30 attendees from across the sector. 

Slides from the session are below:

A broad-ranging introduction, we covered metadata standards and publishing, through to how this work feeds into national energy digitalisation programmes.

We have applied the following principles in designing/selecting the metadata standards and supporting glossaries:

• Coverage: the standards should provide for appropriate breadth and depth of coverage of the sector
• Flexibility: adopters should be able to select those parts of the standards most relevant to their needs
• Extensibility: the standards should provide a framework within which additional standards can be included as new requirements and use cases arise
• Interoperability: the standards should maximise the interoperability of data between systems and actors
• Reuse: the standards should adopt/reuse existing standards wherever possible
• Consistency: the standards should support adopters to apply the standards as consistently as possible.
• The “metadata standard” is not a single standard, but a collection of core standards and links to others

Example feedback across four themes included:

1. Feasibility, maturity and blockers

“The ability to pass through Licensing seamlessly is critical. Our end users need the ability to access data and know that the licensing is taken care of.

Data needs to be de-siloed from regulated energy ‘monopolies’. Policy and regulation which compels data siol owners to make available their data without cost or undue restrictions is critical to the development of the energy sector. “ 

“Challenges I see in this industry: Data quality, lack of standards application (where they exist (e.g. CIM) and standards in some areas. Lack of metadata capture. Open data triage process is a good starting point but will need adaptation and adoption in each organisation. Potential liability if data is used for purposes that it was not intended for. Others getting commercial gain from ‘my data’ without me being compensated.“ 

2. Flexibility, extensibility, evolution

“Critical for take up. I think the introduction of ‘recommended’ subset of fields will be useful but making it all mandatory could be a barrier for some to participation so lets everyone move collectively forward and improve over time”

“The standards should be evolved completely out in the open (which would be a major departure for the UK energy industry!).  The standards should be developed on GitHub, just like standards like DCAT are discussed on GitHub.”

3. Interoperability

“Everything is an API, no exceptions. We use OpenAPI/Swagger definitions to publish available data sets and methods both internally and externally.  We would really appreciate not having to reinvent the wheel every time. For instance, let’s all standardise on UTC, let’s all publish in JSON,CSV and XML as a minimum. Etc.”

“As a minimum I think consistent application (in meaning) of the mandatory fields across different organisations would be a start.”

4. Benefits, Values, Incentives, Drivers

“Allowing more people to use data has the potential to lead to an improved consumer experience – greater variety of offers and business models”

“Descriptors on subject; format; quality and rights will be useful but has been done more ad hoc; I think we have room for improvement here so anticipate other fields may be useful but we haven’t industrialised our learning yet.

I imagine consistency in standards will be important for partners and also for example researchers/ agencies seeking to understand a ‘joined up picture’ and take a lens to particular use cases or topics “

Finally, we presented the metadata progress from MEDA and its alignment with EDVP. Feedback emphasised the importance of tooling and the potential role as a gold standard best practice guide for publishing open data.  

To build any information service, you’ll need to:

1. Search for energy datasets and energy assets;
2. Access those datasets;
3. Interpret each dataset;
4. Join multiple datasets together.

This sequence of four steps should take a few seconds: Finding energy data & information about assets should be as easy as a web search.

But, today, these four steps require an incredible amount of tedious ‘manual’ work and so often take months to complete (and sometimes it’s simply impossible to complete these steps). This friction costs money and time, inhibits innovation, and slows the rate at which we can reduce carbon emissions to net-zero.

In this document, we’ll look at a single technical framework that could help solve all these problems: linked data.

Overview of the web of energy data

• Data & metadata files will reside on data custodians’ web servers.
• A search engine will provide an entry point into the ‘web of energy data’.
  • The search engine will construct a ‘knowledge graph’ of the web of energy data by crawling the metadata files.
  • This is far more than ‘just’ a list of assets and datasets! It will understand the relationships and the meaning of the data, hence enabling:
  • highly specific searches
  • discovery of data & assets by walking the knowledge graph
• A Directory (adapted from Open Banking) will associate parties with security credentials. This will limit access to shared data: data which is available only to specific groups of people, for specific purposes. For example, access controls might be necessary for personally identifiable information; or commercially sensitive information; or information that could be used to disrupt the energy system.
• Every thing (substation, dataset, energy company, etc.) will have a unique ID.
• This ID will take the form of a web address (URL).
• Accessing that web address will return metadata describing the thing.
• The metadata will be in a standard format and use a standard vocabulary; and will link to related resources (just as web pages link to other web pages).
• Taken together, these linked metadata files will form a distributed graph (‘network’) representing the energy system itself and energy datasets; and their relationships.
• Metadata will be compatible with search engines, so users can also enter the “web of energy data” through conventional search engines.
• Open data (as distinct from metadata) can be published in any machine-readable format. But there may be recommended formats for new datasets.
• A standard API will be defined to allow data providers to restrict access to shared data (data that is sensitive or requires payment).

Benefits

Just like the world wide web, content on the ‘web of energy data’ will be:

• No “big-bang” IT projects: Allow industry to adapt piece-by-piece.
• Distributed: Organisations host data on their own servers so will retain full control. No single point of failure. Scales rapidly.
• Easy to interpret: Metadata will use a common technical language.
• Easy to create: The metadata standard will be as simple as possible, and as similar as possible to existing vocabularies.
• Uniquely identifiable: Hence trivial to combine. Unique identifiers for energy assets & datasets is the first step towards combining existing energy asset registers.
• Linked to related assets.
• Searchable & Discoverable through the Knowledge Graph.
• Secured using a Directory adapted from Open Banking.

What does success look like?

• Substantially reduced costs and time to publish and consume energy data.
• Spawn a vibrant ecosystem of innovative energy information services.
• Data and assets will be easy to find as searching the web for a local restaurant.
• Enable an energy system capable of orchestrating millions of DERs.
• Hence help reduce carbon emissions to net-zero without breaking the bank! (The bit that Open Climate Fix is most passionate about!)

Principles

• Enable the rest of the ecosystem: Don’t compete.
• Collaborate deeply.
• All code & research will be open.
• Build on existing work. Don’t reinvent the wheel.
• Engage with existing industry groups.

Linked data for the energy system

Linked data is an evolution of the world wide web.

The web is a distributed, linked documentation system. Web pages are primarily written for people. But it turns out that machines have a hard time understanding content that’s written for human consumption. To understand natural language you have to know a lot about how the world works, including how people think. Machines struggle with that.

This wasn’t a problem when the web was small. But, as the web grew, users needed help searching and processing information on the web. Early automated search tools often produced unsatisfactory results because they struggled to understand the web’s content.

Today, increasingly large numbers of web sites solve this problem using ‘linked data’: metadata in a standard format which makes content easy for machines to understand. Linked data makes it easier to build automated tools to help us search and process vast quantities of distributed data.

Linked data is sometimes called ‘Web 3.0’ and is the brainchild of Sir Tim Berners-Lee (the inventor of the web). Domains such as biomedical science have used linked data for years to build very powerful, distributed representations of knowledge.

In Sir Tim Berners-Lee’s 2009 TED talk on Linked Data, he gave three “extremely simple” principles behind linked data. Here are those three principles applied to energy data:

1. Every thing (dataset, transformer, meter, energy company, etc.) gets a unique name. Each name is a URL (a web address).
2. If I take one of these URLs and I look it up then I get a description of that thing. That description will be in a standard format and will use standard naming conventions.
3. That description also includes relationships by linking to URLs of related things. For example, a transformer would link to the meters attached to that transformer, and those meters would link to descriptions of time series datasets recorded from those meters, and those descriptions would link to the data.

What would a ‘web of energy data’ look like?

Maybe the National Grid ESO would publish linked data describing the transmission system. For example, each grid supply point (GSP) could be identified by a unique web address (e.g. NationalGridESO.com/GSP#NEWX_1). Accessing that web address would return a machine-readable file describing that GSP and linking to descriptions of the supergrid transformers that make up the GSP (see Figure below). To make life as easy as possible for the ESO, this data would be converted from the ESO’s existing CIM models.

The DNOs could publish descriptions of the distribution system. These descriptions would be machine-readable and use a common standard, hence making it trivial to combine models from all DNOs into a description of the entire distribution system.

In the figure below, the rectangles represent metadata files published on the ESO and a DSO’s website. Note that the DSO doesn’t need to create their own representation of each GSP. Instead, the DSO’s metadata just links to the ESO’s description of each GSP.

DSR aggregators could publish metadata describing their portfolios and pricing, etc.

So we’d end up with lots of metadata files (each describing a thing) scattered across the web. Great. But, um, how do users find anything?!

A search engine for linked energy data

On the web, we use search engines to find information. So let’s build a search engine for linked energy data. But we don’t want just any old search engine…

Remember that the proposed metadata files describe relationships: they’re linked. In computer-science-speak, we’d say the files constitute a ‘distributed graph’, where ‘graph’ means a network of nodes (things) connected by edges (the relationships between things). So, to exploit the full power of the web of energy data, we need a search engine that can capture these relationships: a ‘knowledge graph’. This would continually read all the metadata files to construct a single ‘graph’ describing the energy system.

This knowledge graph will provide far more than just a catalogue of datasets and assets! The knowledge graph will understand the relationships between things; and will understand the meaning of the descriptions. So users will be able to run queries like:

• List all the solar PV systems on constrained parts of the network.
• List all wind farms which openly publish time series datasets recording their outturn.
• List all energy assets, along with their geographical location (to preserve privacy, domestic assets would only show their approximate location)

These queries should return results in a fraction of a second, even though the source data is distributed across many systems.

Note that we don’t need to build a ‘full’ search engine: The knowledge graph will only crawl energy metadata. It won’t attempt to crawl all the linked data on the web! Nor will it store any datasets: The knowledge graph will only copy the metadata.

There are several off-the-shelf graph databases available: We don’t need to invent much technology; we mostly just have to bolt together existing technology.

How does the knowledge graph find metadata to crawl? To start with, people will manually enter the web addresses of descriptions of energy things. From that first metadata file, the crawler will follow links to other metadata files. As the metadata becomes more interconnected, the crawler will be able to find more and more metadata on its own.

The knowledge graph may perform other functions: it could track search statistics to allow the community to measure interest in particular datasets; it could validate the metadata it crawls to automatically alert data owners when metadata needs to be fixed; and it could associate datasets with documentation, discussion and tools.

(The most famous ‘knowledge graph’ is probably Google’s knowledge graph. You see results from Google’s knowledge graph when Google shows an ‘infobox’ with search results.)

Just as there are multiple web search engines, there could be multiple ‘energy knowledge graphs’. The source data is distributed and open. And, once users know what data they need, they can go straight to the data source without going through the knowledge graph.

See the appendix for more information on the knowledge graph for energy data & assets.

How to build a linked web of energy data

1. Agree on a common technical language for describing energy assets
   1. First, collect the requirements for a common language by surveying existing datasets and talking to the community. What concepts must the language be able to describe?
   2. Then survey existing metadata schemas & ontologies. Do any of them satisfy all the requirements?
   3. If not: Start sketching out a new schema:
   4. in intimate collaboration with the industry,
   5. and building from existing schemas where possible.
   6. Where appropriate, engage with existing industry bodies.
   7. Focus on real-world use-cases: Don’t try to describe the entire universe!
   8. The metadata should be compatible with existing search engines.
   9. Keep it simple! Unnecessary complexity has killed many linked data projects in the past.
   10. Figure out the technical & organisational structures to enable the community to evolve the standard over time.
2. Convert some existing datasets to the new format. e.g.
   1. Renewable Energy Planning Database (REPD)
   2. System Wide Resource Register (SWRR)
   3. Embedded Capacity Register (ECR)
   4. (Once these conversions are done, these datasets will be crawlable by Google Dataset Search)
3. Work with existing data providers to help them convert their public data to the new format.
4. Build a ‘minimal viable product’ of the knowledge graph in the cloud using an off-the-shelf graph database.
5. Build tools to make it as easy as possible to contribute to the ‘linked web of energy data’. Example tools might include:
   1. Energy companies are very comfortable using Excel. So let’s build a web tool to convert metadata descriptions created in Excel to the format used by the ‘linked web of energy data’.
   2. A ‘wizard’ to author metadata from scratch.
   3. Validators to check metadata.

Unique identifiers are essential

Unique IDs are absolutely fundamental to information management. Without Unique IDs, users can’t automatically combine datasets or follow links to related assets.

Imagine running a large international company if the only identifier you have for each employee is their nickname. It would be impossible! Each identifier (nickname) could be associated with thousands of individuals. There’d be no way to distinguish between all the “Jacks”. Even trivial operations like joining two database tables would be impossible (e.g. one table giving date of birth for each nickname; and another table giving address for each nickname). Companies fix this problem by assigning each employee a unique ID.

Today, we have multiple energy asset registers in the UK (e.g. FiT & REPD). These are extremely hard to combine because they do not use universal, unique IDs. Even within some energy companies, different IT systems use different names for the same physical bits of infrastructure! We need to fix this! We need unique IDs. Once we have unique IDs, we can then map existing asset registers onto the unique names, and hence build a complete asset register.

Enable users to search across assets and datasets at the same time

Energy assets and energy datasets are intimately linked. Data describes assets. Assets create data. An ‘asset registry’ and a ‘dataset registry’ are just two views of the same physical system. To interpret a dataset correctly, users need to understand the assets that produced the data. To understand the behaviour of an asset, users need to find data associated with that asset.

As such, the linked metadata will link assets and datasets; and the knowledge graph will allow users to quickly identify datasets created by assets, and identify assets associated with datasets.

How does this make it easier to build an asset register?

Maybe, like today, the energy system of the future wouldn’t have a single asset register: organisations would continue to publish data for assets they’re responsible for.

But, unlike today, all asset registers would be trivial to combine because they’d all use the same identifiers, and they’d use the same ‘language’ to describe assets, and they’d be linked. And there’d be (at least one) knowledge graph that would automatically combine and link all the asset registers together.

So, from a users’ perspective, it should feel like there’s just one asset register: the knowledge graph. But, from a data publishers’ perspective, they get to keep total control over the data that describes their ‘patch’ of the energy system.

A dynamic asset register

Imagine an asset register which not only describes the ‘static’ properties of each asset, but also describes some ‘dynamic’ properties. For example, gas-fired power stations could report their real-time carbon intensity updated every minute, to help refine the Carbon Intensity API’s calculations. Linked data should make a ‘dynamic’ asset register possible: Power stations would update a public metadata file, which links to an ‘official’ asset register, and provides a live view of the asset.

Sensitive metadata

Maybe we don’t want to openly publish the full details of domestic assets. For example, home owners may not want to broadcast to the world that they have an expensive battery installed in their garage.

The public metadata description of domestic assets could just describe an approximate geographical location. The public metadata could link to restricted metadata that would give authorised users the exact location.

Layers of the Energy Onion

The energy system is becoming harder to manage as it becomes more complex. David Sykes’s fascinating Energy Onion article gives insight as to how to manage this complexity: by considering the energy system as a sequence of layers:

(Image from David’s blog)

This is reminiscent of the way software engineers manage complexity by building layers of abstraction. e.g. Operating systems abstract away the complexity of the hardware. (Or, an even geekier analogy: the TCP/IP networking stack). Maybe David’s layers of the energy onion could be thought of as layers of abstraction?

Inner layers would provide services to the layer immediately above. For example, services which optimise a building’s energy assets to maximise self consumption would be entirely responsible for managing assets within each building. This layer would provide services to the next layer: local balancing and cooperation. These two layers would communicate in both directions. But, crucially, the local balancing layer wouldn’t try to micromanage individual assets. Instead, it would (politely) ask the building’s energy management system to ramp up / ramp down.

The key to managing complexity is to ensure that outer layers don’t micromanage inner layers: An outer layer can’t directly control an inner layer’s assets. Instead, outer layers must go through the inner layer’s interface.

This way of thinking about the energy system also helps inform our priorities for defining standards for data sharing: The priority is to define data standards for the communication between layers. The communication within layers is an ‘implementation detail’ that outer layers don’t need to worry about. (Also see this Vox article: Renewable energy threatens to overwhelm the grid. Here’s how it can adapt.)

How does this fit in with the billion existing energy data projects?

Ha! Good question. And, there really are a lot of existing energy data projects (which is very exciting… but also just a little, um, overwhelming!)

Broadly, the approach isn’t to compete with existing projects. Rather the aim is to enable future & ongoing projects; and to learn from recently completed projects; and to collaborate intimately with the community.

To list a few existing projects:

Open Energy (this initiative)

• A Modernising Energy Data Access (MEDA) project.
• Led by Icebreaker One, in collaboration with PassivSystems & Open Climate Fix.
• MEDA projects finished phase 1 on 2020-07-10
• MEDA projects are currently in Phase 2.
• Icebreaker One is mostly focused on solving the business challenges associated with sharing energy data, but may also have a working group looking at data standards.
• This group is building a ‘linked web of energy data’ with MEDA phase 2 funding

National Digital Twin

We’re very keen to ensure the ‘linked web of energy data’ is compatible with the National Digital Twin, to make it easy to build a ‘twin’ that includes the energy system and the built environment.

Data portals (Flexr, ESO Data Portal, etc.)

The ambition would be to work with Flexr & ESO & others to ensure the ‘common technical language’ is capable of expressing all the concepts these portals need to express, and then to help these portals expose metadata using the ‘common technical language’.

Common Information Model (CIM)

• CIM is already used by parts of the electricity system (e.g. transmission system operators) to describe both the structure of the transmission system, and its dynamical state.
• Officially adopted by the International Electrotechnical Commission (IEC).
• CIM is related to RDF, and RDF is one of the foundations for linked data.
• The linked web of energy data should be able to represent all the concepts already expressed in CIM, to make it easy to convert between CIM and the linked web of energy data. e.g. ESO would just run their existing CIM model through a converter to publish a model compatible with the linked web of energy data.
• On its own, CIM may not be sufficient to represent the entire energy system because:
  • CIM doesn’t describe the grid edge,
  • CIM only describes electricity, not gas,
  • CIM objects can’t be linked over the Internet: A DNO can’t link to the ESO’s definition of a GSP, for example,
  • It’s challenging to push new CIM ‘profiles’ upstream, and so different energy companies end up defining different ‘profiles’ for the same concepts. Hence breaking the main point of having a common information model

ENA’s Digital System Map

• Hopefully the ‘web of linked energy data’ should make it super-easy to build a system map :). The ‘web of linked energy data’ would represent all the map data in a standardised way; and the map UI could be built using an off-the-shelf mapping tool.

Open Energy Ontology

• The OEO is very related!
• Maybe the OEO will be the one ‘magic bullet’ that can describe the energy system or, if not, maybe the OEO can be extended.
• At present, the OEO is focused on energy system modelling, and so can describe the essentials of an energy system (transformers, generators, etc) but doesn’t yet appear to capture some of the implementation details of a real energy system (e.g. GSPs and BSPs; and that transformers might have multiple meters, each with different characteristics, etc.)

Web of Things

• A W3C linked data standard for the Internet of Things
• “Each thing is identified by a URI that can be dereferenced to obtain a machine interpretable description. The URI can also be used as an identifier in rich descriptions as a basis for semantic interoperability between suppliers and consumers of services. Thing descriptions expressed in JSON-LD cover the behavior, interaction affordances, data schema, security configuration and protocol bindings.“

Linux Foundation Energy – Open Energy Data Initiative

• Open energy datasets curated on a single cloud platform
• Open-source tools to access & analyse the datasets

HyperCat

• “HyperCat is an open, lightweight JSON-based hypermedia catalogue format for exposing collections of URIs. Each HyperCat catalogue may expose any number of URIs, each with any number of RDF-like triple statements about it. HyperCat is simple to work with and allows developers to publish linked-data descriptions of resources. HyperCat is designed for exposing information about IoT assets over the web. It allows a server to provide a set of resources to a client, each with a set of semantic annotations. Implementers are free to choose or invent any set of annotations to suit their needs. A set of best practices and tools are currently under development. Where implementers choose similar or overlapping semantics, the possibilities for interoperability are increased“
• GitHub repos haven’t been updated since 2016

SpecialPrivacy.EU

• *”The SPECIAL project (Scalable Policy-aware Linked Data Architecture For Privacy, Transparency and Compliance) addresses the contradiction between Big Data innovation and data protection compliance requirements by proposing a technical solution that makes the achievement of both of these goals realistic. SPECIAL allows citizens and organisations to share more data, while guaranteeing compliance with data protection, thus enabling both trust and the creation of valuable new insights from shared data.”*
• 3 year project. Ended at the end of 2019.

For more details, please see the appendix.

This post is cross-posted from Jack’s blog at Open Climate Fix