Philipp Heer

Buildings consume 30% and 40% of the end-use energy in Europe and worldwide respectively, but the increasing amount of sensors installed in new or retrofitted buildings gives rise to new data-driven control opportunities. Most controllers in existing buildings are however still rule-based, due to the required engineering to develop more advanced methods, such as Model Predictive Control (MPC). On the other hand, Reinforcement Learning (RL) has received increasing attention in the past years as a decision making paradigm that bypasses the need for models. Instead, an agent learns to take optimal decisions solely by interacting with the environment and getting rewarded/penalized for it.

We conjecture that incorporating prior physics-based knowledge in the model architectures or learning procedures will be a stepping stone towards generally applicable - i.e. data-efficient and reliable - DRL approaches. In this work, we will thus explore various solutions to introduce prior knowledge in models and DRL agents, design algorithms to apply them to buildings, and deploy the resulting controllers on case studies to assess their advantages in practice, primarily focusing on the building energy systems installed at the Empa NEST demonstrator.

Electrical, thermal and chemical processes follow individual demand and supply patterns under distinct time scales, from sub-second (electrical) to weekly (thermal) and seasonal (chemical) applications. In these applications, digitalization enables a coordination of the related technologies. So, next to sector specific technological limitations, the systemic coupling of energy carriers needs to take into account the coupling of different time scales as well as different production and consumption patterns. As the technological landscape is getting more decentral in terms of energy production, also more stakeholders are involved in the energetic supply chain. These actors can benefit if they share information on their capabilities and intended production and consumption. Moreover, information sharing and local decision making could potentially reduce the need for extending large-scale infrastructure, such as transmission grids or international imports of energy. The project investigates the use of distributed decision making and control methods to address the large scale nature of the problem, as well as data driven methods to deal with the uncertainty inherent in the problem. Validation of the methodological approach is foreseen on the NEST building at Empa.

Philipp Heer

Scientific Publications

Research projects

Research projects as Researcher