A review of reinforcement learning for autonomous building energy management

Mason, Karl; Grijalva, Santiago

doi:10.1016/j.compeleceng.2019.07.019

Cited by 225 publications

(81 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, because reinforcement learning is computationally demanding, how to better integrate the computation platforms with the building management system is also important for the application of RL in buildings. Mason and Grijalva (2019) [26] reviewed the application of reinforcement learning for building energy management, including HVAC, water heater, appliances, lighting, photovoltaics (PV), batteries and the electrical grid. It was found that RL can typically provide savings of about 10% for HVAC and about 20% for water heaters.…”

Section: Previous Reviewsmentioning

confidence: 99%

Reinforcement learning for building controls: The opportunities and challenges

2020

View full text Add to dashboard Cite

Building controls are becoming more important and complicated due to the dynamic and stochastic energy demand, on-site intermittent energy supply, as well as energy storage, making it difficult for them to be optimized by conventional control techniques. Reinforcement Learning (RL), as an emerging control technique, has attracted growing research interest and demonstrated its potential to enhance building performance while addressing some limitations of other advanced control techniques, such as model predictive control. This study conducted a comprehensive review of existing studies that applied RL for building controls. It provided a detailed breakdown of the existing RL studies that use a specific variation of each major component of the Reinforcement Learning: algorithm, state, action, reward, and environment. We found RL for building controls is still in the research stage with limited applications (11%) in real buildings. Three significant barriers prevent the adoption of RL controllers in actual building controls: (1) the training process is time consuming and data demanding, (2) the control security and robustness need to be enhanced, and (3) the generalization capabilities of RL controllers need to be improved using approaches such as transfer learning. Future research may focus on developing RL controllers that could be used in real buildings, addressing current RL challenges, such as accelerating training and enhancing control robustness, as well as developing an opensource testbed and dataset for performance benchmarking of RL controllers.

show abstract

Section: Previous Reviewsmentioning

confidence: 99%

Reinforcement learning for building controls: The opportunities and challenges

2020

View full text Add to dashboard Cite

show abstract

“…Although the above-mentioned works are effective, there are two major drawbacks in training a DRL agent. Firstly, it is impractical to let the DRL agent to explore the state space fully in a real building environment since unacceptably high cost may be incurred [35] [46] [47]. Secondly, it may take a long time for the DRL agent to learn an optimal policy if trained in a real-world environment [46] [47].…”

Section: Applications Of Drl In a Single Buildingmentioning

confidence: 99%

Deep Reinforcement Learning for Smart Home Energy Management

Xie

et al. 2020

IEEE Internet Things J.

307

View full text Add to dashboard Cite

In this paper, we investigate an energy cost minimization problem for a smart home in the absence of a building thermal dynamics model with the consideration of a comfortable temperature range. Due to the existence of model uncertainty, parameter uncertainty (e.g., renewable generation output, nonshiftable power demand, outdoor temperature, and electricity price) and temporally-coupled operational constraints, it is very challenging to determine the optimal energy management strategy for scheduling Heating, Ventilation, and Air Conditioning (HVAC) systems and energy storage systems in the smart home. To address the challenge, we first formulate the above problem as a Markov decision process, and then propose an energy management strategy based on Deep Deterministic Policy Gradients (DDPG). It is worth mentioning that the proposed strategy does not require the prior knowledge of uncertain parameters and building thermal dynamics model. Simulation results based on real-world traces demonstrate the effectiveness and robustness of the proposed strategy.

show abstract

“…Qolomany et al (2019) [4] and Djenouri et al (2019) [5] reviewed how machine learning and big data could be applied to smart buildings. Vázquez-Canteli & Nagy (2019) [6] and Mason, K. & Grijalva (2019) [7] reviewed how reinforcement learning, a subdomain of machine learning, could be used to enhance building control. Saha et al (2019) [8] reviewed how data analytics could be used for occupancy sensing in buildings.…”

Section: Introductionmentioning

confidence: 99%

State-of-the-art on research and applications of machine learning in the building life cycle

Hong

Wang

Luo

et al. 2020

Energy and Buildings

248

View full text Add to dashboard Cite

Fueled by big data, powerful and affordable computing resources, and advanced algorithms, machine learning has been explored and applied to buildings research for the past decades and has demonstrated its potential to enhance building performance. This study systematically surveyed how machine learning has been applied at different stages of building life cycle. By conducting a literature search on the Web of Knowledge platform, we found 9579 papers in this field and selected 153 papers for an in-depth review. The number of published papers is increasing year by year, with a focus on building design, operation, and control. However, no study was found using machine learning in building commissioning. There are successful pilot studies on fault detection and diagnosis of HVAC equipment and systems, load prediction, energy baseline estimate, load shape clustering, occupancy prediction, and learning occupant behaviors and energy use patterns. None of the existing studies were adopted broadly by the building industry, due to common challenges including (1) lack of large scale labeled data to train and validate the model, (2) lack of model transferability, which limits a model trained with one data-rich building to be used in another building with limited data, (3) lack of strong justification of costs and benefits of deploying machine learning, and (4) the performance might not be reliable and robust for the stated goals, as the method might work for some buildings but could not be generalized to others. Findings from the study can inform future machine learning research to improve occupant comfort, energy efficiency, demand flexibility, and resilience of buildings, as well as to inspire young researchers in the field to explore multidisciplinary approaches that integrate building science, computing science, data science, and social science.

show abstract

A review of reinforcement learning for autonomous building energy management

Cited by 225 publications

References 68 publications

Reinforcement learning for building controls: The opportunities and challenges

Reinforcement learning for building controls: The opportunities and challenges

Deep Reinforcement Learning for Smart Home Energy Management

State-of-the-art on research and applications of machine learning in the building life cycle

Contact Info

Product

Resources

About