MARCO - Multi-Agent Reinforcement learning based COntrol of building HVAC systems

Nagarathinam, Srinarayana; Menon, Vishnu; Vasan, Arunchandar; Sivasubramaniam, Anand

doi:10.1145/3396851.3397694

Cited by 36 publications

(20 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Comfort is modeled as a setpoint temperature like 23 • C, and the method aims to optimize energy efficiency as long as the indoor temperature is not drifted far from the setpoint. Similar works can also be found in [3]- [6]. However, such empirical assumption on comfort has proven to be inaccurate in many cases, and as a result, the energy saving is often achieved with the significant sacrifice of occupant comfort.…”

mentioning

confidence: 62%

Toward Intelligent Multizone Thermal Control With Multiagent Deep Reinforcement Learning

Zhang

Gao

et al. 2021

IEEE Internet Things J.

View full text Add to dashboard Cite

Energy usage and thermal comfort are the pillars of smart buildings. Many research works have been proposed to save energy while maintaining a comfortable thermal condition. However, most of them either make the over-simplified assumption on thermal comfort with unsatisfied comfort performance or deal with the single-zone thermal control only with limited practical impact. A few preliminary pieces of research on multi-zone control are available, but they fail to keep pace with the latest advancements in the deep learning-based control techniques. In this paper, we investigate the multi-zone thermal control with optimized energy usage and canonical thermal comfort modeling. We adopt the emerging multi-agent deep reinforcement learning techniques and propose to model each zone as an agent. A multi-agent framework is established to support the information exchange among the agents and enable intelligent thermal control in the heterogeneous zones. Accordingly, we mathematically formulate a problem to optimize both energy and comfort. A multizone thermal control algorithm (MOCA) is proposed to solve the problem by deriving optimal control policies. We validate the performance of MOCA through simulation in professional TRNSYS, configured based on our real-world laboratory. The results are promising with up to 15.4% energy-saving as well as satisfied thermal comfort in different zones.

show abstract

mentioning

confidence: 62%

Toward Intelligent Multizone Thermal Control With Multiagent Deep Reinforcement Learning

Zhang

Gao

et al. 2021

IEEE Internet Things J.

View full text Add to dashboard Cite

show abstract

“…The authors in [45] propose a methods for the optimal scheduling of different household appliances to optimize energy utilization. The authors in [16] propose a MARL algorithm to minimize HVAC energy consumption without sacrificing user comfort by adjusting both the building and chiller set-points. To speed up the training process, they use transfer learning in which the agents are trained on sub-sets of HVAC systems and the learned network weights are used to initialize the multiple agents.…”

Section: B Deep Rl Methods In Bemsmentioning

confidence: 99%

Building Energy Management With Reinforcement Learning and Model Predictive Control: A Survey

Zhang¹,

Seal²,

Wu³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Building energy management has been recognized as one of the core problems in modern power grids with respect to system operation efficiency. However, the building energy management system (BEMS) is now facing more challenges and uncertainties with the increasing penetration of renewable energy and increasing adoption of different types of electrical appliances and equipment. Classical model predictive control (MPC) has shown effective in building energy management, although it suffers from labour-intensive modelling and complex online control optimization. Recently, with the growing accessibility to building control and automation data, data-driven solutions such as data-driven MPC and reinforcement learning (RL)-based methods have attracted more research interest. However, the potential of integrating these two types of methods and how to choose suitable control algorithms have not been well discussed. In this work, we first present a compact review of the recent advances in data-driven MPC and RL-based control methods for building energy management. Furthermore, the main challenges in these approaches and general discussions on the selection of control methods are discussed.INDEX TERMS Building energy management, model predictive control, reinforcement learning, datadriven control.

show abstract

“…In [64], Morinibu et al proposed a A2C-based HVAC control method t uniformity of radiation temperature in the room. In [26], Nagarathinam et al proposed a multi-agent D to minimize HVAC energy consumption without sacrificing user comfort by adjusting both the building an To be specific, each DDQN-based agent coordinate with each other to learn an optimal HVAC control p proposed. For example, Wei et al [15] proposed a DQN-based HVAC control method to save energy cost in office buildings while maintaining the room temperature requirements.…”

Section: Applications Of Drl In a Single Buildingmentioning

confidence: 99%

“…In [64], Morinibu et al proposed a A2C-based HVAC control method to decrease the non-uniformity of radiation temperature in the room. In [26], Nagarathinam et al proposed a multi-agent DRL based algorithm to minimize HVAC energy consumption without sacrificing user comfort by adjusting both the building and chiller set-points. To be specific, each DDQN-based agent coordinate with each other to learn an optimal HVAC control policy.…”

Section: Applications Of Drl In a Single Buildingmentioning

confidence: 99%

Deep Reinforcement Learning for Smart Home Energy Management

Xie

et al. 2020

IEEE Internet Things J.

308

View full text Add to dashboard Cite

In this paper, we investigate an energy cost minimization problem for a smart home in the absence of a building thermal dynamics model with the consideration of a comfortable temperature range. Due to the existence of model uncertainty, parameter uncertainty (e.g., renewable generation output, nonshiftable power demand, outdoor temperature, and electricity price) and temporally-coupled operational constraints, it is very challenging to determine the optimal energy management strategy for scheduling Heating, Ventilation, and Air Conditioning (HVAC) systems and energy storage systems in the smart home. To address the challenge, we first formulate the above problem as a Markov decision process, and then propose an energy management strategy based on Deep Deterministic Policy Gradients (DDPG). It is worth mentioning that the proposed strategy does not require the prior knowledge of uncertain parameters and building thermal dynamics model. Simulation results based on real-world traces demonstrate the effectiveness and robustness of the proposed strategy.

show abstract

MARCO - Multi-Agent Reinforcement learning based COntrol of building HVAC systems

Cited by 36 publications

References 26 publications

Toward Intelligent Multizone Thermal Control With Multiagent Deep Reinforcement Learning

Toward Intelligent Multizone Thermal Control With Multiagent Deep Reinforcement Learning

Building Energy Management With Reinforcement Learning and Model Predictive Control: A Survey

Deep Reinforcement Learning for Smart Home Energy Management

Contact Info

Product

Resources

About