Drawing a line from data to better grid management

As the field of automation accelerates, one of the most exciting areas for real-world innovation is energy management. Better automation algorithms can bring dramatic improvements in grid resilience, efficiency and flexibility – all urgently needed. But none of those benefits can be achieved without advances in the underlying theoretical understanding. That’s where Dr. Mengshuo Jia (Shuo) is focusing his attention.    

In studying power systems, the fundamental models that describe the grid are too complex to be useful, especially in large-scale systems. This problem is commonly addressed by linearising, rendering a model that can be manipulated and used for optimisation, whether for industrial applications or research. These linear models are now integral to the daily operations, market clearing, and grid planning of power systems around the world, unlocking trillion-dollar markets.  

Data-driven methods bring new perspectives – and new questions 

Linearisation is not new, but new opportunities are emerging with the development of data-driven models.  

The research on physics-based linearisation approaches for power systems has been developed for decades. But these methods rely on the physical parameters of the grid: the characteristics of power lines, how they are connected, the control rules of privately owned energy resources, and so on. Determining these precise values is harder than you might think! They fluctuate, for example, according to factors such as temperature, the age of the power cables, and more. That makes it expensive and difficult (or even impossible) to obtain the values needed, especially in distribution grids (the networks of wires and poles you may see along city streets).   

In contrast, data-driven linearisation approaches offer far more flexibility, and usually, far more accuracy. Instead of the exact physical parameters of the grid at a given time, only the operating data is needed, which is collected automatically and continuously by measurement units deployed across the grid. These methods also allow for uncertainties, while offering scope for customisation, without relying on the assumptions commonly demanded by physics-driven methods.   

But in the context of power systems, there are now dozens of data-driven linearisation approaches to choose from. How can anyone know which is the best one for their needs? When new models are published, only a selective comparison is made to show their benefits (never a full evaluation against all other available methods), and since they usually aren’t open-source, it’s hard to assess how they work.    

To fill this gap, Shuo has conducted theoretical and numerical evaluations of each data-driven method, examining how it works, what it is good at, where it might fall short, and how it can be improved. This evaluation covered 40 data-driven methods as well as four traditional physics-driven ones, measuring them across dozens of tests to see how they actually perform in terms of accuracy, speed and more.   

This provides reproducible evidence to support researchers and engineers in determining the best method for their tasks, but the project’s significance is about accessibility as well as selection. Unlike the physics-based methods, which are either open-source or built into common software, the new data-driven approaches are not necessarily easily available. “It’s important to make these new methods easier for everyone to use, not just specialists in the data-driven domain,” argues Shuo. “This way, more people can take advantage of the latest improvements to enhance their own projects.”  

Easy toolkit for academia and industry 

To address this need, Shuo has developed a unique toolbox. Daline (named for data-driven linearisation) incorporates more than 50 linearisation methods, including 10 new proposals as well as the plethora of data-driven models from the literature and several classical physics-based models. But the toolbox goes further, enabling data generation, cleaning, processing, and visualising as well as testing and comparison of the various approaches.  

Daline was created to help users handle complex tasks using just a few, simple commands – most users will need to learn just three. Beneath the user-friendly surface is a complex system, with hundreds of files supporting a range of modules with more than 300 adjustable parameters. The toolkit comes with an extensive manual (over 150 pages), created by Shuo and his research assistant Wen Yi Chan. But if that sounds daunting, take heart: Shuo also developed AgentDaline, leveraging large language models (LLMs), to further increase accessibility. Imagine AgentDaline as an automated assistant (like J.A.R.V.I.S. from the Iron Man movies) that you can message in natural language, asking it to clarify the use of various parameters and demonstrate Daline functions. More importantly, it can also do simulation tasks using Daline, correct coding bugs if any, and return simulation results back to users. 

Building AgentDaline meant establishing a general framework that enables LLMs to perform power system simulations on request, using toolboxes like Daline, even though these AI models have never seen these toolboxes before. "Enabling LLMs to run power system simulations (like Daline) might signal a new research paradigm, making these AI models effective research assistants for human scientists,” says Shuo. 

How can the toolbox promote a sustainable grid? 

In practical terms, better models mean better planning. Armed with more accurate data, power grid operators can make better day-to-day decisions (which generators to run and how much power to generate?) and devise better long-term strategies. Given that the best models available through Daline are often over three orders of magnitude more accurate than the most commonly used models, DC or PTDF, the toolbox has major implications for the management of the national power grid. It becomes possible to ensure far more reliable and efficient unit commitment; and since there is no need to spend more money to determine the physical parameters, it is also more widely applicable.   

Of course, looking ahead, we can immediately see a major reason to celebrate this development in power flow linearisation. Renewable energy sources introduce far more uncertainty, and more fluctuations, to the power grid. That really underlines the limitations of the physics-driven linearisation models. As renewables take a greater role in energy provision, the complexity of grid management is soaring. Operators will need data-driven models to accommodate that complexity. They will need to develop or identify a model that is general enough to accommodate all the fluctuations that renewables introduce, and since historical data (from a grid with less reliance on renewables) will rapidly be outdated, they will need to continuously retrain their model based on current data. Luckily, Daline makes that easy.   

Swissgrid collaboration puts Daline to work 

The tools have already been seen in action in Shuo’s collaboration with Swissgrid. Together they used data-driven linearisation methods to support remedial actions for a European power grid, encompassing more than 11,000 connection points and 18,000 power lines, in order to improve grid reliability when incorporating renewables. The results, which show the superiority of data-driven methods, will be shared at the IEEE Power & Energy Society General Meeting in July 2024 by Aline Scherrer (a student Shuo has supervised).    

Daline is a practical product of highly academic work on linearisation. Both the exhaustive underlying evaluation of all available methods, and the user-friendly tools themselves, serve the goal of providing fundamental research and tools to assist other researchers as well as engineers. The work has been demanding, but has brought unexpected discoveries along the way. “In developing the whole toolbox, I have identified and solved a problem in that field. And because of that, I also developed several new methods,” Shuo says. “Plenty of unsolved problems, and new methods for solving them, have turned up in my research.”  

One particular insight secured Shuo a grant from the Swiss National Science Foundation, funding him to investigate the linearity mechanism in power systems. This is a purely theoretical topic associated with many aspects of the energy domain. He hopes this work, like Daline, will result in foundational insights that can benefit the field at large. “It’s like playing a game, but my aim isn’t to win,” says Shuo. “I want to enable others to play better.”    

References: 

[1]  Daline: https://www.shuo.science/daline.

[2]  Mengshuo Jia, Gabriela Hug, “Overview of Data-driven Power Flow Linearization,” 2023 IEEE PowerTech Conference, Belgrade, Serbia. 

[3]  Mengshuo Jia, Gabriela Hug, Ning Zhang, Zhaojian Wang, Yi Wang, Chongqing Kang. “Data-driven Power Flow Linearization: Theory”.  

[4]  Mengshuo Jia, Gabriela Hug, Ning Zhang, Zhaojian Wang, Yi Wang, Chongqing Kang. “Data-driven Power Flow Linearization: Simulation”.  

[5]  Mengshuo Jia, Wen Yi Chan, Gabriela Hug. “Daline: A Data-driven Power Flow Linearization Toolbox for Power Systems Research and Education,” ETH Research Collection.   

[6] Mengshuo Jia, Zeyu Cui, Gabriela Hug. “Enabling Large Language Models to Perform Power System Simulations with Previously Unseen Tools: A Case of Daline”. 

[7]  Mengshuo Jia, Wen Yi Chan, Gabriela Hug. “User Manual for Daline 1.1.5,” ETH Research Collection.  

[8]  Aline Scherrer, Georgia Pierrou, Mengshuo Jia, Marc Hohmann, Gabriela Hug. “Physics-Data-Driven Power Flow Linearization Considering Topological Remedial Actions,” IEEE PES General Meeting 2024, Seattle, WA, USA. 

[9]  Mengshuo Jia, “Rethinking Power Systems Computation: Uncovering Linearity/Nonlinearity Mechanism,” Swiss National Science Foundation, Principal Investigator, No.221126.