Reinforcement Learning: Theory and Applications in HEMS

Abstract

The steep rise in reinforcement learning (RL) in various applications in energy as well as the penetration of home automation in recent years are the motivation for this article. It surveys the use of RL in various home energy management system (HEMS) applications. There is a focus on deep neural network (DNN) models in RL. The article provides an overview of reinforcement learning. This is followed with discussions on state-of-the-art methods for value, policy, and actor–critic methods in deep reinforcement learning (DRL). In order to make the published literature in reinforcement learning more accessible to the HEMS community, verbal descriptions are accompanied with explanatory figures as well as mathematical expressions using standard machine learning terminology. Next, a detailed survey of how reinforcement learning is used in different HEMS domains is described. The survey also considers what kind of reinforcement learning algorithms are used in each HEMS application. It suggests that research in this direction is still in its infancy. Lastly, the article proposes four performance metrics to evaluate RL methods.

Imitation Learning with Deep Attentive Tabular Neural Networks for Environmental Prediction and Control in Smart Home

“…In reinforcement learning terminology, an agent is a learning entity that exerts control over a stochastic external environment by means of a sequence of actions over time [7]. The agent learns to improve the performance of its environment using reward signals that are derived from the latter.…”

“…A recent article [7] has provided a comprehensive exposition of reinforcement learning as well as an in-depth survey of its various smart home applications. In reinforcement learning for indoor environmental control, the reward signal 𝑟 is based directly on comfort indices that may not be suitable.…”

“…Various kinds of indoor appliances in a smart home can be controlled in this manner (cf. [6,7]), such as lighting, refrigeration, PHEV charging, battery storage, water pumps, and interior environmental control.…”

“…Reinforcement learning is an emerging machine learning paradigm that allows an Energies 2023, 16, 5091 2 of 19 algorithm to learn from experience (cf. [7]). In recent years, deep neural networks (DNNs) have emerged as the most popular machine learning models for a wide range of applications [8].…”

Interoperability Testing of a Smart Home Automation System under Explicit Demand Response Schemes

“…The topic of smart home and home energy management have attracted the scientific interest with several articles on the subject. In [3], an energy management system (EMS) is suggested for residential microgrids in order to minimize their operational cost. In [4], an EMS is proposed for efficient use of power generated by a smart home, where every building and appliance have their own controller.…”