Energy systems

The electric energy demand of buildings is increasing due to the electrification of the transportation and heating systems. While this puts an additional strain on the power grid, EVs and heat pumps (among others) are flexible loads that can be leveraged to keep the balance between generation and load. Coincidently, smart meters and edge computing resources (i.e. building automation systems) are deployed providing the means to access this flexibility. Hence, in this project, we will leverage building data to develop detailed models for the quantification and usage of flexibility at the building level for different time scales.

Buildings consume 30% and 40% of the end-use energy in Europe and worldwide respectively, but the increasing amount of sensors installed in new or retrofitted buildings gives rise to new data-driven control opportunities. Most controllers in existing buildings are however still rule-based, due to the required engineering to develop more advanced methods, such as Model Predictive Control (MPC). On the other hand, Reinforcement Learning (RL) has received increasing attention in the past years as a decision making paradigm that bypasses the need for models. Instead, an agent learns to take optimal decisions solely by interacting with the environment and getting rewarded/penalized for it.

We conjecture that incorporating prior physics-based knowledge in the model architectures or learning procedures will be a stepping stone towards generally applicable - i.e. data-efficient and reliable - DRL approaches. In this work, we will thus explore various solutions to introduce prior knowledge in models and DRL agents, design algorithms to apply them to buildings, and deploy the resulting controllers on case studies to assess their advantages in practice, primarily focusing on the building energy systems installed at the Empa NEST demonstrator.

The provision of ancillary services is an important aspect to ensure the stability and the functioning of the electric power system, even more so in a system with increasing levels of renewable generation. However, the type of ancillary services and how they are being compensated likely needs to be adjusted from the current practice, which was designed for highly centralized energy production and inflexible demand. In this project, we will identify the type of services needed, how the capabilities of the individual resources can optimally be used and how they should be remunerated for their services.

The project addresses the question of optimal system layout for facilitating decentralized control and optimal operation of a network of power converters with generators, storages and loads. Special emphasis is placed on the system capability of operating in a decentralized fashion to meet specific local control objectives (e.g. keeping the local voltage within prescribed limits) and system-wide control objectives (e.g. fair proportional load sharing among the inverters, provision of ancillary services). Sensitivity analysis will enable to define the boundaries of a completely decentralized structure and where adding a hierarchical level enables a significant performance improvement.

With the replacement of synchronous machine based generation by power electronics connected renewable generators, the speed of frequency dynamics increases and new voltage issues arise in distribution systems due to decreased system inertia and the distributed nature of these resources. In this project, we focus on the provision of ancillary services by distributed energy resources (DER) at different time scales. This entails Fast Frequency Control provided by grid-forming Voltage Source Converters and at the slower time scale, the integration of active distribution grids in a set of ancillary services.

We study how flexible devices, consumers and producers can participate in an automated fashion in an energy market that yields efficient market outcomes. As the availability of distributed resources in terms of production and flexibility is uncertain, probabilistic approaches for the automated bidding but also for the determination of the required reserves are necessary. Given that distributions of the willingness of the participants to accept prices are unknown, we intend to formulate and explore distributionally robust optimization models.

Cyberphysical systems are composed by subsystems linked through physical interconnections and communication channels enabling distributed control architectures. By leveraging recent results on optimal control, we will develop distributed algorithms for the data-driven generation of networked regulators adapted to the communication topology. Special attention will be devoted to the decentralization of the design process, so as to facilitate the addition and removal of subsystems in a plug-and-play fashion. As benchmark examples, we will consider energy systems and manufacturing processes, which are key applications in the NCCR.

In spite of several success stories, standard deep networks  can lead to unstable forward propagation or suffer from an ill-posed learning process, which might result in vulnerability to adversarial attacks. In this project we will study deep network architectures based on ODEs for mitigating these problems. Furthermore, we will investigate the use of these networks for parametrizing regulators that can be tuned from available data. The new methods will be tested for the control of inverters, smart loads and microgrid clusters, which are relevant for the energy systems considered in the NCCR.

The electrification of transportation forecast for the near future will couple two problems of similar nature that have been studied independently until now: congestion on the road network and congestion on the electric network. In this project, we will study the viability of coordinated incentives that exploit this coupling to simultaneously ease these congestions more effectively than independently designed incentives.

We will investigate the theoretical and practical challenges of using energy harvesting to power nodes distributed control systems. Combining energy sources such as temperature and vibration with battery systems and wireless links enables the placement of sensor nodes where they are needed for the best data quality, regardless of the availability of wired power or communication. In the context of the NCCR, we will jointly investigate some of the open challenges in the design of autonomous, energy-neutral automation systems. In a second phase, we will investigate suitable demonstrators and applications such as wireless sensing and control in motor control and energy systems.

State Estimation is an important tool in the Energy Management systems of electric power systems to monitor the state of the system. It takes measurements from the field and usually by the means of weighted least square optimization identifies the most likely system state. This step is necessary as measurements are subject to measurement errors. Traditionally, the resulting problem formulation includes the non-linear power flow equations. However, in previous work, we have derived a formulation that allows for a linear representation exhibiting the same accuracy but clearly superior computational performance to the non-linear approaches. In this project, we propose to explore how such linear formulation provides computationally efficient means to identify and filter false data injection (FDI) attacks. Protecting the power system against FDI attacks is of critical importance as many control functionalities rely on the reported state of the system.

This project will study formal methods for reduced complexity design and verification of embedded optimization techniques for the control of high-speed nonlinear constrained systems. We will focus on machine learning techniques for differentiable parametric optimization and their application to the control of fast, nonlinear dynamic systems with a power conversion system taken as an important exemplar case study.

Electrical, thermal and chemical processes follow individual demand and supply patterns under distinct time scales, from sub-second (electrical) to weekly (thermal) and seasonal (chemical) applications. In these applications, digitalization enables a coordination of the related technologies. So, next to sector specific technological limitations, the systemic coupling of energy carriers needs to take into account  the coupling of different time scales as well as different production and consumption patterns. As the technological landscape is getting more decentral in terms of energy production, also more stakeholders are involved in the energetic supply chain. These actors can benefit if they share information on their capabilities and intended production and consumption. Moreover, information sharing and local decision making could potentially reduce the need for extending large-scale infrastructure, such as transmission grids or international imports of energy. The project investigates the use of distributed decision making and control methods to address the large scale nature of the problem, as well as data driven methods to deal with the uncertainty inherent in the problem. Validation of the methodological approach is foreseen on the NEST building at Empa.

For temporally varying electric distribution networks, reliable information about the system topology and parameters may be missing or outdated. In the project we will develop online estimation algorithms for the network reconstruction, so as to automatically track changes, even in the presence of noisy and incomplete data. Furthermore, we will blend online estimation algorithms with higher-level controllers commonly used in distribution networks  and apply our methods to the moon-shot testbed developed in the NCCR. Our ultimate goal is to contribute to the development of autonomous electric networks that self-adapt in real-time to changing ambient conditions.

Recent results on data-based control exploit alternative parametrizations of linear and nonlinear systems, such as those provided by behavioral theory and deep neural networks. In this project we focus on Input/Output Behavioral (IOB) models that have been used for control by assuming noiseless measurement or deterministic bounded disturbances. The goal of the project is to study the relation between IOB-based control methods and the existing approaches for stochastic systems, including system identification and stochastic model predictive control. New controllers will be tested on the energy systems relevant for the NCCR.

The project aims at developing an autonomous and reconfigurable control architecture where each converter operates in plug-and-play mode, i.e., without additional communication overhead or detailed model knowledge, and under variable system conditions. This requires in a first step that each converter estimates online the equivalent circuit as seen at its terminals. Based on such estimates, the converter controller reconfigures its structure and adapts online its parameters in order to optimally perform under the different grid configurations and operating conditions.

Recent years have seen a surge of academic and industrial interest in distributionally robust optimization, where the probability distribution of the uncertain problem parameters is itself uncertain and one seeks decisions that are optimal in view of the most adverse distribution within a given ambiguity set. In this project we will study online distributionally robust optimization with streaming data. The key motivation is that dynamic stochastic processes (as encountered in control, estimating, and filtering problems) demand recursive and online solutions with real-time computational constraints. We plan to implement our approaches on various energy system platforms.

Online feedback optimization refers to the design of feedback controllers that asymptotically steer a physical system to the solution of an optimization problem while respecting physical and operational constraints. Here we are interested in exploring self-interested agents that do not want to cooperate for the sake of achieving a common goal but first and foremost have their own interest in mind. A relevant real-world example is selfish and uncoordinated congestion control by different power transmission system operators. We will investigate distributed Nash-seeking algorithms to solve the resulting antagonistic decision-making problems, and also deploy them in numerical and real-world case studies.

We investigate how classical methods from system identification, dynamical systems, and control can be combined with techniques from non- parametric machine learning to develop a rigorous framework for continuous-time model-based deep reinforcement learning. Concretely, we will learn a continuous-time dynamics model via neural ordinary differential equations, along with associated uncertainty bounds that can be used to trade exploration and exploitation. Ultimately, our framework should enable control design for complex and uncertain systems, and we plan to apply it to problems in building automation and grid-connected converters.