ADP-based optimal sensor scheduling for target tracking in energy harvesting wireless sensor networks

Abstract

This paper proposes a novel sensor scheduling scheme based on adaptive dynamic programming, which makes the sensor energy consumption and tracking error optimal over the system operational horizon for wireless sensor networks with solar energy harvesting. Neural network is used to model the solar energy harvesting. Kalman filter estimation technology is employed to predict the target location. A performance index function is established based on the energy consumption and tracking error. Critic network is developed to approximate the performance index function. The presented method is proven to be convergent. Numerical example shows the effectiveness of the proposed approach.

Adaptive Dynamic Programming-Based Multi-Sensor Scheduling for Collaborative Target Tracking in Energy Harvesting Wireless Sensor Networks

“…Up to now, ADP has been applied to nonlinear zero-/nonzero-sum differential games [48,49], optimal tracking control problems [50], optimal control of intelligent grid [51,52], and optimal time slot scheduling of MAC protocol [53]. Recently ADP was also proposed as an optimal sensor scheduling scheme for target tracking in an energy-harvesting WSN [20], by scheduling one sensor for each time step over an infinite horizon considering the global tracking accuracy and energy consumption. However, ADP-based multi-sensor scheduling for collaborative target tracking in energy-harvesting WSNs remains as an open and challenging problem.…”

“…Characterized by strong abilities of self-learning and adaptation, ADP has demonstrated a strong capability to find the optimal control policy and solve the discrete system dynamic programming (DP) problem [18], including the adaptive critic design, reinforcement learning [19], and so on, obtaining the approximate optimal performance and the optimal control to satisfy the Bellman optimal principle through the function approximation structure. An ADP-based sensor scheduling scheme for target tracking in an energy-harvesting WSN was proposed in [20], which made the sensor energy consumption and tracking accuracy optimal over the system operational horizon for WSNs. However, only one sensor was scheduled for each time step, and therefore the tracking accuracy improvement was limited.…”

Online reinforcement learning for a class of partially unknown continuous‐time nonlinear systems via value iteration

“…Hence, to tackle this problem, the value iteration method is combined with the IRL scheme to design a novel controller.Belonging to the machine learning community, NNs are used as the approximators in the ADP analysis; thus, the method is referred to as the neurodynamic programming. [53][54][55][56] In this work, a critic design scheme is employed in estimating the cost function and control laws. Only the critic NN needs to be turned; the control policy can be estimated directly by the cost function obtained from the critic NN.…”