Towards optimal control of air handling units using deep reinforcement learning and recurrent neural network

Zou, Zhengbo; Yu, Xinran; Ergan, Semiha

doi:10.1016/j.buildenv.2019.106535

Cited by 127 publications

(46 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In model-based methods, DRL agents need to learn building environment models based on historical data, e.g., MuZero [45], Long Short-Term Memory-Deep Deterministic Policy Gradients (LSTM-DDPG) [46], differentiable MPC policy-Proximal Policy Optimization (differentiable MPC policy-when updating the weights , which can improve the stability of training process. Furthermore, the weights of the target network are updated every steps in Line 13, which makes the DQN algorithm more stable.…”

Section: Drl Classificationmentioning

confidence: 99%

“…In model-based methods, DRL agents need to learn building environment models based on historical data Long Short-Term Memory-Deep Deterministic Policy Gradients (LSTM-DDPG) [46], differentiable mode policy-Proximal Policy Optimization (differentiable MPC policy-PPO) [47]. [63], Advantage [64], Asynchronous Advantage Actor-Critic (A3C) [65]), and maximum entropy methods (e.g., Multi-Ac (MAAC) [17], Entropy-Based Collective Advantage Actor-Critic (EB-C-A2C) [27], Entropy-Based Collecti (EB-C-DQN) [27]).…”

Section: Applications Of Drl In a Single Buildingmentioning

confidence: 99%

“…Firstly, it is impractical to let the DRL agent to explore the state space fully in a real building environment since unacceptably high cost may be incurred [35] [46] [47]. Secondly, it may take a long time for the DRL agent to learn an optimal policy if trained in a real-world environment [46] [47]. To reduce the dependency on a real building environment, many modelbased DRL control methods have been developed [46] [65].…”

Section: Applications Of Drl In a Single Buildingmentioning

confidence: 99%

“…Secondly, it may take a long time for the DRL agent to learn an optimal policy if trained in a real-world environment [46] [47]. To reduce the dependency on a real building environment, many modelbased DRL control methods have been developed [46] [65]. For example, Zhang et al [65] proposed and implemented a Building Energy Model (BEM)-based DRL control framework for a novel radiant heating system in an existing office building.…”

Section: Applications Of Drl In a Single Buildingmentioning

confidence: 99%

“…Since building energy models are calibrated based on the observed data in a short period of time (e.g., three months), the DRL agent's exposure to real-world HVAC operational data is limited [65]. To overcome this drawback, Zou et al [46] proposed a DRL-based HVAC control framework to minimize energy consumption while maintaining thermal comfort levels for occupants based on operational data within two years. The proposed control framework is composed of two parts as shown in Fig.…”

Section: Applications Of Drl In a Single Buildingmentioning

confidence: 99%

See 4 more Smart Citations

Deep Reinforcement Learning for Smart Home Energy Management

Xie

et al. 2020

IEEE Internet Things J.

308

View full text Add to dashboard Cite

In this paper, we investigate an energy cost minimization problem for a smart home in the absence of a building thermal dynamics model with the consideration of a comfortable temperature range. Due to the existence of model uncertainty, parameter uncertainty (e.g., renewable generation output, nonshiftable power demand, outdoor temperature, and electricity price) and temporally-coupled operational constraints, it is very challenging to determine the optimal energy management strategy for scheduling Heating, Ventilation, and Air Conditioning (HVAC) systems and energy storage systems in the smart home. To address the challenge, we first formulate the above problem as a Markov decision process, and then propose an energy management strategy based on Deep Deterministic Policy Gradients (DDPG). It is worth mentioning that the proposed strategy does not require the prior knowledge of uncertain parameters and building thermal dynamics model. Simulation results based on real-world traces demonstrate the effectiveness and robustness of the proposed strategy.

show abstract

Section: Drl Classificationmentioning

confidence: 99%

Section: Applications Of Drl In a Single Buildingmentioning

confidence: 99%

Section: Applications Of Drl In a Single Buildingmentioning

confidence: 99%

Section: Applications Of Drl In a Single Buildingmentioning

confidence: 99%

Section: Applications Of Drl In a Single Buildingmentioning

confidence: 99%

See 3 more Smart Citations

Deep Reinforcement Learning for Smart Home Energy Management

Xie

et al. 2020

IEEE Internet Things J.

308

View full text Add to dashboard Cite

show abstract

A practically implementable reinforcement learning‐based process controller design

Hassanpour,

Wang,

Corbett

et al. 2023

AIChE Journal

View full text Add to dashboard Cite

The present article enables reinforcement learning (RL)‐based controllers for process control applications. Existing instances of RL‐based solutions have significant challenges for online implementation since the training process of an RL agent (controller) presently requires practically impossible number of online interactions between the agent and the environment (process). To address this challenge, we propose an implementable model‐free RL method developed by leveraging industrially implemented model predictive control (MPC) calculations (often designed using a simple linear model identified via step tests). In the first step, MPC calculations are used to pretrain an RL agent that can mimic the MPC performance. Specifically, the MPC calculations are used to pretrain the actor, and the objective function is used to pretrain the critic(s). The pretrained RL agent is then employed within a model‐free RL framework to control the process in a way that initially imitates MPC behavior (thus not compromising process performance and safety), but also continuously learns and improve its performance over the nominal linear MPC. The effectiveness of the proposed approach is illustrated through simulations on a chemical reactor example.

show abstract

Intelligent Multi-zone Residential HVAC Control Strategy Based on Deep Reinforcement Learning

Li,

2023

Power Electronics and Power Systems

View full text Add to dashboard Cite

Towards optimal control of air handling units using deep reinforcement learning and recurrent neural network

Cited by 127 publications

References 42 publications

Deep Reinforcement Learning for Smart Home Energy Management

Deep Reinforcement Learning for Smart Home Energy Management

A practically implementable reinforcement learning‐based process controller design

Intelligent Multi-zone Residential HVAC Control Strategy Based on Deep Reinforcement Learning

Contact Info

Product

Resources

About