RLMan: An Energy Manager Based on Reinforcement Learning for Energy Harvesting Wireless Sensor Networks

Abstract

A promising solution to achieve autonomous wireless sensor networks is to enable each node to harvest energy in its environment. To address the time-varying behavior of energy sources, each node embeds an energy manager responsible for dynamically adapting the power consumption of the node in order to maximize the quality of service while avoiding power failures. A novel energy management algorithm based on reinforcement learning, named RLMan, is proposed in this work. By continuously exploring the environment, RLMan adapts its energy management policy to time-varying environment, regarding both the harvested energy and the energy consumption of the node. Linear function approximations are used to achieve very low computational and memory footprint, making RLMan suitable for resource-constrained systems such as wireless sensor nodes. Moreover, RLMan only requires the state of charge of the energy storage device to operate, which makes it practical to implement. Exhaustive simulations using real measurements of indoor light and outdoor wind show that RLMan outperforms current state of the art approaches, by enabling almost 70 % gain regarding the average packet rate. Moreover, RLMan is more robust to variability of the node energy consumption.

Autonomous Management of Energy-Harvesting IoT Nodes Using Deep Reinforcement Learning

“…They simulated a communication environment and approximated the state space with discrete features that indicate battery level and its constraints, harvested energy over an hour, characteristics of the communication link, data arrival process, and the data buffers at the communicating nodes. Similarly, Aoudia et al [6], used an actor-critic method with linear function approximation to learn approximation for both policy and value function. They used a Gaussian policy to generate continuous values of bounded packet rates and summarized the state space by continuous values of the current residual energy.…”

The Smaller the Better: Designing Solar Energy Harvesting Sensor Nodes for Long-Range Monitoring

“…In order to enable sustainable operation of IoT nodes, energy harvesting technologies have been considered for a long time by improving both hardware components and platforms and the associated software methods to properly manage the energy consumption [4,8,9]. These software components are often referred to as energy managers and are not addressed in this study.…”

“…This parameter is set to 2 days per week without any illuminance. Moreover, by applying (8), the supercapacitor needs to be set to 148.87 mF in order to absorb the current peaks of the transmission process (40.9 mA for of 14 dBm). Results show that increasing from 1 mn to 60 mn induces a battery reduction factor of 41 for SL setup, of 4.5 for SD setup and 1.3 for SH setup.…”

Novel wireless charging algorithms to charge mobile wireless sensor network by using reinforcement learning

“…Khelladi et al [5] proposed multi-node charging where the numbers of sensors charged at every stop for energy transfer. Aoudia et al [6] proposed a novel RL man algorithm based upon reinforcement learning for the energy conservation of the sensor nodes. This method gained almost 70% of the average packet rate.…”