Towards plug&amp;play smart thermostats inspired by reinforcement learning

Marantos, Charalampos; Lamprakos, Christos P.; Tsoutsouras, Vasileios; Siozios, Kostas; Soudris, Dimitrios

doi:10.1145/3285017.3285024

Cited by 9 publications

(5 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to penalizing temperature deviations from a setpoint, frequent switching of the on/off heating element can be penalized if it is perceived as annoying to the occupant, due to the noise involved, for example [88]. An alternative approach to binary control is to allow the RL agent to add or subtract a fixed value from the temperature setpoint [89][90][91]. A double binary control approach involves all the possible combinations of on/off control of two heating elements in a hot water tank [92].…”

Section: Overview Of the Analyzed Articlesmentioning

confidence: 99%

A Review of Reinforcement Learning Applications to Control of Heating, Ventilation and Air Conditioning Systems

2022

View full text Add to dashboard Cite

Reinforcement learning has emerged as a potentially disruptive technology for control and optimization of HVAC systems. A reinforcement learning agent takes actions, which can be direct HVAC actuator commands or setpoints for control loops in building automation systems. The actions are taken to optimize one or more targets, such as indoor air quality, energy consumption and energy cost. The agent receives feedback from the HVAC systems to quantify how well these targets have been achieved. The feedback is captured by a reward function designed by the developer of the reinforcement learning agent. A few reviews have focused on the reward aspect of reinforcement learning applications for HVAC. However, there is a lack of reviews that assess how the actions of the reinforcement learning agent have been formulated, and how this impacts the possibilities to achieve various optimization targets in single zone or multi-zone buildings. The aim of this review is to identify the action formulations in the literature and to assess how the choice of formulation impacts the level of abstraction at which the HVAC systems are considered. Our methodology involves a search string in the Web of Science database and a list of selection criteria applied to each article in the search results. For each selected article, a three-tier categorization of the selected articles has been performed. Firstly, the applicability of the approach to buildings with one or more zones is considered. Secondly, the articles are categorized by the type of action taken by the agent, such as a binary, discrete or continuous action. Thirdly, the articles are categorized by the aspects of the indoor environment being controlled, namely temperature, humidity or air quality. The main result of the review is this three-tier categorization that reveals the community’s emphasis on specific HVAC applications, as well as the readiness to interface the reinforcement learning solutions to HVAC systems. The article concludes with a discussion of trends in the field as well as challenges that require further research.

show abstract

Section: Overview Of the Analyzed Articlesmentioning

confidence: 99%

A Review of Reinforcement Learning Applications to Control of Heating, Ventilation and Air Conditioning Systems

2022

View full text Add to dashboard Cite

show abstract

“…An online version of FQI that uses a neural network, Neural Fitted Q-Iteration, was proposed by [26]. Marantos et al [27] applied NFQ batch RL to control the thermostat set-point of a single-zone building where input state was fourdimensional (outdoor and indoor temperatures, solar radiance, and indoor humidity) and action was one-dimensional with three discrete values.…”

Section: Rl With Action-value Function Approximationmentioning

confidence: 99%

Data-driven control of micro-climate in buildings: An event-triggered reinforcement learning approach

et al. 2020

View full text Add to dashboard Cite

“…The authors reported energy savings in the range of 4 -11%, comparable to the previous study. [70]. This approach also used a neural network to approximate the Q function.…”

Section: Hvacmentioning

confidence: 99%

A review of reinforcement learning for autonomous building energy management

Mason

Grijalva

2019

Computers & Electrical Engineering

228

View full text Add to dashboard Cite

The area of building energy management has received a significant amount of interest in recent years. This area is concerned with combining advancements in sensor technologies, communications and advanced control algorithms to optimize energy utilization. Reinforcement learning is one of the most prominent machine learning algorithms used for control problems and has had many successful applications in the area of building energy management. This research gives a comprehensive review of the literature relating to the application of reinforcement learning to developing autonomous building energy management systems. The main direction for future research and challenges in reinforcement learning are also outlined.

show abstract

Towards plug&play smart thermostats inspired by reinforcement learning

Cited by 9 publications

References 27 publications

A Review of Reinforcement Learning Applications to Control of Heating, Ventilation and Air Conditioning Systems

A Review of Reinforcement Learning Applications to Control of Heating, Ventilation and Air Conditioning Systems

Data-driven control of micro-climate in buildings: An event-triggered reinforcement learning approach

A review of reinforcement learning for autonomous building energy management

Contact Info

Product

Resources

About