Minimizing AoI in Resource-Constrained Multi-Source Relaying Systems with Stochastic Arrivals

Abstract

We consider a multi-source relaying system where the sources independently and randomly generate status update packets which are sent to the destination with the aid of a bufferaided relay through unreliable links. We formulate a stochastic optimization problem aiming to minimize the sum average age of information (AAoI) of sources under per-slot transmission capacity constraints and a long-run average resource constraint.To solve the problem, we recast it as a constrained Markov decision process (CMDP) problem and adopt the Lagrangian method. We analyze the structure of an optimal policy for the resulting MDP problem that possesses a switching-type structure. We propose an algorithm that obtains a stationary deterministic near-optimal policy, establishing a benchmark for the system. Simulation results show the effectiveness of our algorithm compared to benchmark algorithms.

On-Demand AoI Minimization in Resource-Constrained Cache-Enabled IoT Networks with Energy Harvesting Sensors

“…AoI-optimal scheduling has attracted a significant amount of interest from the research community over the last few years [4]- [27]. Particularly, a popular approach is to model the problem as an MDP and find an optimal policy by using model-based reinforcement learning (RL) methods based on dynamic programming [4], [5], [9]- [12], [15], [16], [18]- [20], [26], [27], e.g., relative value iteration algorithm (RVIA), and/or model-free RL methods [4], [9], [10], [14], [21], [22], e.g., (deep) Q-learning.…”

“…In [11], the authors proposed an asymptotically optimal algorithm for the AoI-optimal scheduling problem under both bandwidth and average power constraints in a wireless network with timevarying channel states. In [12], the authors studied AoI minimization problem in a multi-source relaying system under per-slot transmission and average resource constraints.…”

“…Different from [5]- [12], another line of research [13]- [21] focused on the class of problems where the sources are powered by energy harvested from the environment, i.e., investigating AoI-optimal scheduling policies subject to the energy causality constraint at the source(s). The works [13]- [20] studied AoI-optimal scheduling in single-sensor EH networks where the sensor is sending time-sensitive information to the user(s).…”

“…We make a common assumption (see e.g., [4], [7], [9], [10], [12], [14]- [16], [18]- [22]) that ∆ k (t) is upper-bounded by a finite value ∆ max , i.e., ∆ k (t) ∈ {1, 2, . .…”