Energy-aware task scheduling in wireless sensor networks based on cooperative reinforcement learning

Abstract

Wireless sensor networks (WSN) are an attractive platform for cyber physical systems. A typical WSN application is composed of different tasks which need to be scheduled on each sensor node. However, the severe energy limitations pose a particular challenge for developing WSN applications, and the scheduling of tasks has typically a strong influence on the achievable performance and energy consumption. In this paper we propose a method for scheduling the tasks using cooperative reinforcement learning (RL) where each node determines the next task based on the observed application behavior. In this RL framework we can trade the application performance and the required energy consumption by a weighted reward function and can therefore achieve different energy/performance results of the overall application. By exchanging data among neighboring nodes we can further improve this energy/performance tradeoff. We evaluate our approach in an target tracking application. Our simulations show that cooperative approaches are superior to non-cooperative approaches for this kind of applications.

Reward Function Evaluation in a Reinforcement Learning Approach for Energy Management

“…Exploitation deals with the actions which have been chosen based on the previously learned utility of the actions. A heuristic is used where the exploration probability at any point of time is given in [10]:…”

Resource-aware task scheduling by an adversarial bandit solver method in wireless sensor networks

“…Khan and Rinner [9] apply cooperative reinforcement learning (CRL) for online task scheduling. They use StateAction-Reward-State-Action (SARSA(λ)) [22] learning and introduced cooperation among neighboring sensor nodes to further improve the task scheduling.…”

“…Khan and Rinner [9] apply SARSA(λ) (CRL), also referred to as State-Action-Reward-State-Action, which is an iterative algorithm that approximates the optimal solution. SARSA(λ) [22] is an iterative algorithm that approximates the optimal solution without knowledge of the transition probabilities which is very important for a dynamic system like WSN.…”

“…The balancing factor of the reward function is varied, and the number of nodes in the network and the randomness of moving targets to find out the tracking quality/energy consumption trade-off are also varied. The average execution time and communication overhead for distributed independent reinforcement learning (DIRL) [7], reinforcement learning (RL) [8], cooperative reinforcement learning (CRL) [9], and Exp3 are also calculated.…”

Throughput-Aware Cooperative Reinforcement Learning for Adaptive Resource Allocation in Device-to-Device Communication

“…where ρ is a positive constant and visited(s, a) represents the visited state-action pairs so far [30] (Step 6 in Algorithm 2). 8 Update eligibility traces 9 Update the Q-value, Q t+1 (s t+1 , a t+1 ) ← Q t (s t , a t ) + αδ t e t (s t , a t ) 10 Update the value function and share with neighbors 11 Shift to the next based on the executed action 12 end loop Algorithm 1 depicts the overall proposed resource allocation method.…”