Dynamic Energy Management for Perpetual Operation of Energy Harvesting Wireless Sensor Node Using Fuzzy Q-Learning

Abstract

In an energy harvesting wireless sensor node (EHWSN), balance of energy harvested and consumption using dynamic energy management to achieve the goal of perpetual operation is one of the most important research topics. In this study, a novel fuzzy Q-learning (FQL)-based dynamic energy management (FQLDEM) is proposed in adapting its policy to the time varying environment, regarding both the harvested energy and the energy consumption of the WSN. The FQLDEM applies Q-learning to train, evaluate, and update the fuzzy rule base and then uses the fuzzy inference system (FIS) for determining the working duty cycle of the sensor of the EHWSN. Through the interaction with the energy harvesting environment, the learning agent of the FQL will be able to find the appropriate fuzzy rules in adapting the working duty cycle for the goal of energy neutrality such that the objective of perpetual operation of the EHWSN can be achieved. Experimental results show that the FQLDEM can maintain the battery charge status at a higher level than other existing methods did, such as the reinforcement learning (RL) method and dynamic duty cycle adaption (DDCA), and achieve the perpetual operation of the EHWSN. Furthermore, experimental results for required on-demand sensing measurements exhibit that the FQLDEM method can be slowly upgraded to meet 65% of the service quality control requirements in the early stage, which outperforms the RL-based and DDCA methods.

Electric Vehicle Charging System in the Smart Grid Using Different Machine Learning Methods

“…The Q-learning algorithm was used by [53] in 2012 to improve the power management system of electric and hybrid bicycles. In terms of power management, these researchers aimed to increase rider comfort and safety and utilize battery power more effectively.…”

“…To find the best way to maintain an HEV's battery at the ideal charge level. The author in [53] used the Q-learning algorithm. By combining this strategy with a long-term plan, you can strike a balance between maximum effectiveness and the capacity to act now.…”

Integration of online deep reinforcement learning and federated learning: A dynamic energy management strategy for industrial applications

“…Additionally, we also find that energy harvesting and management occupy a crucial role in federated learning [25], particularly on energy-limited devices [26][27][28][29]. However, existing research has not fully and deeply considered the impact of energy management on federated learning, especially when dealing with the dynamics and uncertainties of federated learning, these issues become more pronounced.…”

Q-Learning and Efficient Low-Quantity Charge Method for Nodes to Extend the Lifetime of Wireless Sensor Networks

“…It is based on iterative offline operations that predict the next optimal step based on obtained experience. Hence, the lifetimes of nodes and WSNs have been extended using Q-learning [18,19], and low power consumption has been achieved via energy management [20,21]. A novel Q-learning-based data-aggregation-aware energy-efficient routing algorithm was proposed in [22].…”