Reinforcement learning for whole-building HVAC control and demand response

Azuatalam, Donald; Lee, Wee Lih; Nijs, Frits de; Liebman, Ariel

doi:10.1016/j.egyai.2020.100020

Cited by 149 publications

(49 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conversely, for γ = 1 the agent will seek to maximise only future rewards. Similarly, the action-value function represents the goodness of taking a certain action At in a certain state St following a specific control policy π [46]. The action value function can be expressed as follows:…”

Section: Reinforcement Learning Overviewmentioning

confidence: 99%

Online Implementation of a Soft Actor-Critic Agent to Enhance Indoor Temperature Control and Energy Efficiency in Buildings

et al. 2021

View full text Add to dashboard Cite

Recently, a growing interest has been observed in HVAC control systems based on Artificial Intelligence, to improve comfort conditions while avoiding unnecessary energy consumption. In this work, a model-free algorithm belonging to the Deep Reinforcement Learning (DRL) class, Soft Actor-Critic, was implemented to control the supply water temperature to radiant terminal units of a heating system serving an office building. The controller was trained online, and a preliminary sensitivity analysis on hyperparameters was performed to assess their influence on the agent performance. The DRL agent with the best performance was compared to a rule-based controller assumed as a baseline during a three-month heating season. The DRL controller outperformed the baseline after two weeks of deployment, with an overall performance improvement related to control of indoor temperature conditions. Moreover, the adaptability of the DRL agent was tested for various control scenarios, simulating changes of external weather conditions, indoor temperature setpoint, building envelope features and occupancy patterns. The agent dynamically deployed, despite a slight increase in energy consumption, led to an improvement of indoor temperature control, reducing the cumulative sum of temperature violations on average for all scenarios by 75% and 48% compared to the baseline and statically deployed agent respectively.

show abstract

Section: Reinforcement Learning Overviewmentioning

confidence: 99%

Online Implementation of a Soft Actor-Critic Agent to Enhance Indoor Temperature Control and Energy Efficiency in Buildings

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Black horizontal line is the nominal frequency and the battery idle state. The battery simulation model was wrapped in custom Python code that implements an interface similar to the environments in the OpenAI Gym collection [52], which has been used in several recent publications on RL applications in the energy domain [53][54][55][56][57][58]. This interface defines the functions reset(S[0]) and step(a).…”

Section: Implementation 41 Enviromentmentioning

confidence: 99%

A Simulation Environment for Training a Reinforcement Learning Agent Trading a Battery Storage

et al. 2021

View full text Add to dashboard Cite

Battery storages are an essential element of the emerging smart grid. Compared to other distributed intelligent energy resources, batteries have the advantage of being able to rapidly react to events such as renewable generation fluctuations or grid disturbances. There is a lack of research on ways to profitably exploit this ability. Any solution needs to consider rapid electrical phenomena as well as the much slower dynamics of relevant electricity markets. Reinforcement learning is a branch of artificial intelligence that has shown promise in optimizing complex problems involving uncertainty. This article applies reinforcement learning to the problem of trading batteries. The problem involves two timescales, both of which are important for profitability. Firstly, trading the battery capacity must occur on the timescale of the chosen electricity markets. Secondly, the real-time operation of the battery must ensure that no financial penalties are incurred from failing to meet the technical specification. The trading-related decisions must be done under uncertainties, such as unknown future market prices and unpredictable power grid disturbances. In this article, a simulation model of a battery system is proposed as the environment to train a reinforcement learning agent to make such decisions. The system is demonstrated with an application of the battery to Finnish primary frequency reserve markets.

show abstract

“…Reinforcement Learning (RL) was developed by Azuatalam et al [33] for scheduling and controlling the HVAC in commercial buildings. This holistic framework acts as a controller for a building model that optimizes the HVAC system for improving the thermal comfort to achieve demand response goals.…”

Section: Literature Reviewmentioning

confidence: 99%

Forecasting of Energy Demands for Smart Home Applications

Bhatt

Danalakshmi²,

Hariharasudan

et al. 2021

Energies

View full text Add to dashboard Cite

The utilization of energy is on the rise in current trends due to increasing consumptions by households. Smart buildings, on the other hand, aim to optimize energy, and hence, the aim of the study is to forecast the cost of energy consumption in smart buildings by effectively addressing the minimal energy consumption. However, smart buildings are restricted, with limited power access and capacity associated with Heating, Ventilation and Air Conditioning (HVAC) units. It further suffers from low communication capability due to device limitations. In this paper, a balanced deep learning architecture is used to offer solutions to address these constraints. The deep learning algorithm considers three constraints, such as a multi-objective optimization problem and a fitness function, to resolve the price management problem and high-level energy consumption in HVAC systems. The study analyzes and optimizes the consumption of power in smart buildings by the HVAC systems in terms of power loss, price management and reactive power. Experiments are conducted over various scenarios to check the integrity of the system over various smart buildings and in high-rise buildings. The results are compared in terms of various HVAC devices on various metrics and communication protocols, where the proposed system is considered more effective than other methods. The results of the Li-Fi communication protocols show improved results compared to the other communication protocols.

show abstract

Reinforcement learning for whole-building HVAC control and demand response

Cited by 149 publications

References 45 publications

Online Implementation of a Soft Actor-Critic Agent to Enhance Indoor Temperature Control and Energy Efficiency in Buildings

Online Implementation of a Soft Actor-Critic Agent to Enhance Indoor Temperature Control and Energy Efficiency in Buildings

A Simulation Environment for Training a Reinforcement Learning Agent Trading a Battery Storage

Forecasting of Energy Demands for Smart Home Applications

Contact Info

Product

Resources

About