Reinforcement Learning for Control of Building HVAC Systems

Raman, Naren Srivaths; Devraj, Adithya M.; Barooah, Prabir; Meyn, Sean

doi:10.23919/acc45564.2020.9147629

Cited by 27 publications

(9 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this manuscript, we will undertake a comparative study focusing on energy efficiency. Previous research has delved into this scenario of energy efficiency in buildings, both with classical models (e.g., [33][34][35]) and quantum models (e.g., [25,26]).…”

Section: Of 27mentioning

confidence: 99%

Brain-inspired agents for Quantum Reinforcement Learning

Andrés,

Cuellar,

Navarro

2024

Preprint

View full text Add to dashboard Cite

In recent years, advancements in brain science and neuroscience have significantly influenced the field of computer science, particularly in the domain of reinforcement learning (RL). Drawing insights from neurobiology and neuropsychology, researchers have leveraged these findings to develop novel mechanisms for understanding intelligent decision-making processes in the brain. Concurrently, the emergence of quantum computing has opened new frontiers in artificial intelligence, leading to the development of quantum machine learning (QML). This study introduces a novel model that integrates Quantum Spiking Neural Networks (QSNN) and Quantum Long Short-Term Memory (QLSTM) architectures, inspired by the complex workings of the human brain. Specifically designed for reinforcement learning tasks in energy-efficient environments, our approach progresses through two distinct stages mirroring sensory and memory systems. In the initial stage, analogous to the brain’s hypothalamus, low-level information is extracted to emulate sensory data processing patterns. Subsequently, resembling the hippocampus, this information is processed at a higher level, capturing and memorizing correlated patterns. We conduct a comparative analysis of our model against existing quantum models such as Quantum Neural Networks and their classical counterparts, elucidating its unique contributions. Through empirical results, we aim to underscore the effectiveness of our approach in optimizing energy consumption in practical scenarios.

show abstract

Section: Of 27mentioning

confidence: 99%

Brain-inspired agents for Quantum Reinforcement Learning

Andrés,

Cuellar,

Navarro

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The learning process is based on the Q function. Given a policy φ for the problem (13), the Q function associated with this policy is defined as (14) where for k ≥ 0 we have x k+1 = F (x k , u k ) and for k ≥ 1 we have u k = φ(x k ). A well known fact is that the optimal policy satisfies [25] φ * (x) = arg min…”

Section: Rl Basics and Proposed Controller A Rl Basicsmentioning

confidence: 99%

“…A number of papers have investigated the use of RL for control of HVAC systems; see [12]- [14] and references therein. In particular, [12], [13] propose RL controllers for air handling units in buildings. Dynamics of AHUs are considerably simpler than that of DCEPs.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning for Optimal Control of a District Cooling Energy Plant

Zhang¹,

Coffman²,

Barooah³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

District cooling energy plants (DCEPs) consisting of chillers, cooling towers, and thermal energy storage (TES) systems consume a considerable amount of electricity. Optimizing the scheduling of the TES and chillers to take advantage of time-varying electricity price is a challenging optimal control problem. The classical method, model predictive control (MPC), requires solving a high dimensional mixed-integer nonlinear program (MINLP) because of the on/off actuation of the chillers and charging/discharging of TES, which are computationally challenging. RL is an attractive alternative to MPC: the real time control computation is a low-dimensional optimization problem that can be easily solved. However, the performance of an RL controller depends on many design choices.In this paper, we propose a Q-learning based reinforcement learning (RL) controller for this problem. Numerical simulation results show that the proposed RL controller is able to reduce energy cost over a rule-based baseline controller by approximately 8%, comparable to savings reported in the literature with MPC for similar DCEPs. We describe the design choices in the RL controller, including basis functions, reward function shaping, and learning algorithm parameters. Compared to existing work on RL for DCEPs, the proposed controller is designed for continuous state and actions spaces.

show abstract

“…One study linking the reinforcement learning approach to PVT systems has taken control of energy flows between the individual components of an energy system showing positive results [33]. Other studies have also assessed the potential of the RL approach for control tasks in various energy systems (e.g., [34][35][36][37][38]). However, some studies show the risk of affecting comfort for occupants of the respective buildings studied.…”

Section: Existing Studiesmentioning

confidence: 99%

Control of a PVT-Heat-Pump-System Based on Reinforcement Learning–Operating Cost Reduction through Flow Rate Variation

John

Kaltschmitt

2022

Energies

View full text Add to dashboard Cite

This study aims to develop a controller to operate an energy system-consisting of a photovoltaic thermal (PVT) system combined with a heat pump, using the reinforcement learning approach to minimize the operating costs of the system. For this, the flow rate of the cooling fluid pumped through the PVT system is controlled. This flow rate determines the temperature increase of the cooling fluid while reducing the temperature of the PVT system. The heated-up cooling fluid is used to improve the heat pump’s coefficient of performance (COP). For optimizing the operation costs of such a system, first an extensive simulation model has been developed. Based on this technical model, a controller has been developed using the reinforcement learning approach to allow for a cost-efficient control of the flow rate. The results show that a successfully trained control unit based on the reinforcement learning approach can reduce the operating costs with an independent validation dataset. For the case study presented here, based on the implemented methodological approach, including hyperparameter optimization, the operating costs of the investigated energy system can be reduced by more than 4% in the training dataset and by close to 3% in the validation dataset.

show abstract

Reinforcement Learning for Control of Building HVAC Systems

Cited by 27 publications

References 11 publications

Brain-inspired agents for Quantum Reinforcement Learning

Brain-inspired agents for Quantum Reinforcement Learning

Reinforcement Learning for Optimal Control of a District Cooling Energy Plant

Control of a PVT-Heat-Pump-System Based on Reinforcement Learning–Operating Cost Reduction through Flow Rate Variation

Contact Info

Product

Resources

About