Reinforcement Learning Framework for Delay Sensitive Energy Harvesting Wireless Sensor Networks

Al-Tous, Hanan; Barhumi, Imad

doi:10.1109/jsen.2020.3044049

Cited by 12 publications

(5 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The predecessor of the PPO is trust region policy optimization (TRPO). PPO draws on the basic ideas of TRPO and importance sampling to approximate the objective function in (23) to obtain the surrogate objective function. The surrogate objective function can be expressed as:…”

Section: Brief Review Of Ppomentioning

confidence: 99%

“…In order to tackle the access and continuous power control problems simultaneously, [22] proposed an RL method based on actor-critic DQN that takes into account both the sum-rate and prediction loss. In [23], RL is used to develop an optimal control policy based on minimum-delay transmission in a multi-hop energy harvesting wireless sensor network. [24] proposed a novel RL method based on value function approximation to tackle the scheduling problem online.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Proximal Policy Optimization (PPO)-Based Resource Allocation for Energy Harvesting Industrial Wireless Sensor

Li,

Xu,

Tang

et al. 2023

Preprint

View full text Add to dashboard Cite

For the purpose of overcoming the challenges of charging wireless sensors in the complicated industrial environment, researchers are concentrating more and more on sensor networks that can harvest energy.This paper looks at a wirelessly powered industrial sensor network where each sensor harvests energy from a specific radio frequency (RF) energy source and uses it to transmit data to a receiver.Two working modes are discussed of in this paper.One is the frequency division multiplexing (FDM) working mode, where the sensor simultaneously transmits data over orthogonal frequency bands while harvesting RF energy.Time division multiplexing (TDM), which divides each time slot into two successive intervals, is the second working mode.Data is transmitted and energy is harvested in the same frequency band, but at distinct intervals.Because the channel condition and energy harvesting process are unpredictable, an efficient resource allocation algorithm is required for the sensors.We propose a novel resource allocation algorithm based on reinforcement learning.The proposed algorithm achieves continuous resource allocation and is applicable for continuous states by using Proximal Policy Optimization (PPO).We also utilize entropy regularization, online normalization of state, reward scaling, and advantage normalization to improve the performance of resource allocation algorithm in real-world scenarios.In both FDM and TDM working modes, the proposed algorithm outperforms the greedy algorithm and random algorithm in terms of long-term throughput, according to the results of numerical simulations.

show abstract

Section: Brief Review Of Ppomentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Proximal Policy Optimization (PPO)-Based Resource Allocation for Energy Harvesting Industrial Wireless Sensor

Li,

Xu,

Tang

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Let's denote the improvement degree of the chosen individual when one LS operator is employed to refine it, which can be determined using Eq. (10).…”

Section: Adaptive Hill Climbingmentioning

confidence: 99%

“…This research work proposes a novel hybrid algorithm to perform the clustering process in wireless sensor networks. The research work is composed of dual efficient algorithms namely the Memetic algorithm to perform the energy-efficient clustering process [10] while the adaptive hill-climbing algorithm is to identify the critical shortest path among the nodes in the Wireless Sensor Networks. The proposed clustering mechanism is proved to be energy efficient as it transfers the data to the neighbor node which is located at the shortest distance.…”

Section: Introductionmentioning

confidence: 99%

Efficient Clustering Using Memetic Adaptive Hill Climbing Algorithm in WSN

Manikandan¹,

Sakthivel²,

Vivekanandhan³

2023

Intelligent Automation &Amp; Soft Computing

View full text Add to dashboard Cite

Wireless Sensor Networks are composed of autonomous sensing devices which are interconnected to form a closed network. This closed network is intended to share sensitive location-centric information from a source node to the base station through efficient routing mechanisms. The efficiency of the sensor node is energy bounded, acts as a concentrated area for most researchers to offer a solution for the early draining power of sensors. Network management plays a significant role in wireless sensor networks, which was obsessed with the factors like the reliability of the network, resource management, energy-efficient routing, and scalability of services. The topology of the wireless sensor networks acts driven factor for network efficiency which can be effectively maintained by performing the clustering process effectively. More solutions and clustering algorithms have been offered by various researchers, but the concern of reduced efficiency in the routing process and network management still exists. This research paper offers a hybrid algorithm composed of a memetic algorithm which is an enhanced version of a genetic algorithm integrated with the adaptive hill-climbing algorithm for performing energy-efficient clustering process in the wireless sensor networks. The memetic algorithm employs a local searching methodology to mitigate the premature convergence, while the adaptive hill-climbing algorithm is a local search algorithm that persistently migrates towards the increased elevation to determine the peak of the mountain (i.e.,) best cluster head in the wireless sensor networks. The proposed hybrid algorithm is compared with the state of art clustering algorithm to prove that the proposed algorithm outperforms in terms of a network life-time, energy consumption, throughput, etc.

show abstract

“…optimal resource allocation, energy-efficient communication, routing selection, etc.) in WSNs are being explored due to its prominence compared to other traditional approaches [9], [10], [11]. RL is an area of machine learning where an intelligent agent takes actions in a simulated environment and learns the optimal policy by trial-and-error process over time.…”

Section: Introductionmentioning

confidence: 99%

Actively managed battery degradation of wireless sensors for structural health monitoring

Nishat

Jeong

et al. 2023

Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems 2023

View full text Add to dashboard Cite

The battery-powered wireless sensor network (WSN) is a promising solution for structural health monitoring (SHM) applications because of its low cost and easy installation capability. However, the long-term WSN operation suffers from various concerns related to uneven battery degradation of wireless sensors, associated battery management, and replacement requirement, and ensuring desired quality of service (QoS) of the WSN in practice. The battery life is one of the biggest limiting factors for long-term WSN operation. Considering the costly maintenance trips for battery replacement, a lack of effective battery degradation management at the system level can lead to a failure in WSN operation. Moreover, the QoS needs to be ensured under various practical uncertainties. Optimal selection with a maximal number of nodes in WSN under uncertainties is a critical task to ensure the desired QoS. This study proposes a reinforcement learning (RL) based framework for active control of the battery degradation at the WSN system level with the aim of the battery group replacement while extending the service life and ensuring the QoS of WSN. A comprehensive simulation environment was developed in a real-life WSN setup, i.e. WSN for a cable-stayed bridge SHM, considering various practical uncertainties. The RL agent was trained under a developed RL environment to learn optimal nodes and duty cycles, meanwhile managing battery health at the network level. In this study, a mode shape-based quality index is proposed for the demonstration. The training and test results showed the prominence of the proposed framework in achieving effective battery health management of the WSN for SHM.

show abstract

Reinforcement Learning Framework for Delay Sensitive Energy Harvesting Wireless Sensor Networks

Cited by 12 publications

References 41 publications

Proximal Policy Optimization (PPO)-Based Resource Allocation for Energy Harvesting Industrial Wireless Sensor

Proximal Policy Optimization (PPO)-Based Resource Allocation for Energy Harvesting Industrial Wireless Sensor

Efficient Clustering Using Memetic Adaptive Hill Climbing Algorithm in WSN

Actively managed battery degradation of wireless sensors for structural health monitoring

Contact Info

Product

Resources

About