Reinforcement Learning-Based School Energy Management System

Abstract

Energy efficiency is a key to reduced carbon footprint, savings on energy bills, and sustainability for future generations. For instance, in hot climate countries such as Qatar, buildings are high energy consumers due to air conditioning that resulted from high temperatures and humidity. Optimizing the building energy management system will reduce unnecessary energy consumptions, improve indoor environmental conditions, maximize building occupant’s comfort, and limit building greenhouse gas emissions. However, lowering energy consumption cannot be done despite the occupants’ comfort. Solutions must take into account these tradeoffs. Conventional Building Energy Management methods suffer from a high dimensional and complex control environment. In recent years, the Deep Reinforcement Learning algorithm, applying neural networks for function approximation, shows promising results in handling such complex problems. In this work, a Deep Reinforcement Learning agent is proposed for controlling and optimizing a school building’s energy consumption. It is designed to search for optimal policies to minimize energy consumption, maintain thermal comfort, and reduce indoor contaminant levels in a challenging 21-zone environment. First, the agent is trained with the baseline in a supervised learning framework. After cloning the baseline strategy, the agent learns with proximal policy optimization in an actor-critic framework. The performance is evaluated on a school model simulated environment considering thermal comfort, CO2 levels, and energy consumption. The proposed methodology can achieve a 21% reduction in energy consumption, a 44% better thermal comfort, and healthier CO2 concentrations over a one-year simulation, with reduced training time thanks to the integration of the behavior cloning learning technique.

AI-big data analytics for building automation and management systems: a survey, actual challenges and future perspectives

“…( 2020 ); Chemingui et al. ( 2020 ) for controlling the building’s HVAC systems for energy and thermal comfort optimization.…”

A Review of Reinforcement Learning Applications to Control of Heating, Ventilation and Air Conditioning Systems

“…A straightforward application of single agent technology involves a school building with 21 zones of three different types: classrooms, offices, laboratories and a gym. There are 12 possible values for the temperature setpoint and 6 possible values for the CO2 setpoint [5], resulting in scalability issues as discussed in Section 4.3.2.1. A sophisticated work addressing such issues involved a multi-zone building, with AHU and VAV models in the environment.…”

“…The environment provides state information as input to the RL agent. The state could consist of measurements such as temperature and CO2 sensor measurements [5]. Based on the state, the agent outputs actions to the environment.…”

A Review of Thermal Comfort in Primary Schools and Future Challenges in Machine Learning Based Prediction for Children

“…The authors Chemingui et al [133] propose a deep reinforcement learning agent to regulate indoor environmental conditions in a school building while enhancing thermal comfort, and minimizing energy consumption. Likewise, using artificial neural network (ANN) based on an integrated model, authors Cho et al [134] and, Duran et al [135] could predict PMV, concentrations of carbon dioxide (CO 2 ) and estimate heating energy demand and indoor overheating degree, respectively.…”