Bufferless and single-buffer queueing systems have recently been shown to be effective in coping with escalated Age of Information (AoI) figures arising in systems with large buffers and FCFS scheduling. In this paper, we propose a numerical algorithm for obtaining the exact distribution of both the AoI and the peak AoI (PAoI) in the bufferless P H/P H/1/1 and P H/P H/1/1/P -LCFS queues as well as the single-buffer M/P H/1/2 and M/P H/1/2 * queues, the latter one involving the replacement of the packet in the queue by the new arrival. The proposed exact models are based on the well-established theory of Markov fluid queues and the numerical algorithms rely on numerically stable and efficient vector-matrix operations. Moreover, the obtained exact distributions are in matrix exponential form making it amenable to calculate the tail distributions and the associated moments straightforwardly. We validate the proposed algorithms with simulations and we also comparatively study the AoI performance of the four queueing systems of interest as a function of the system load as well as the squared coefficient of variation (scov) of the service time. A similar study is also pursued for assessing the impact of the scov of the interarrival time for the two bufferless queueing systems.

We consider a single-hop wireless network with sources transmitting time-sensitive information to the destination over multiple unreliable channels. Packets from each source are generated according to a stochastic process with known statistics and the state of each wireless channel (ON/OFF) varies according to a stochastic process with unknown statistics. The reliability of the wireless channels is to be learned through observation. At every time slot, the learning algorithm selects a single pair (source, channel) and the selected source attempts to transmit its packet via the selected channel. The probability of a successful transmission to the destination depends on the reliability of the selected channel. The goal of the learning algorithm is to minimize the Age-of-Information (AoI) in the network over T time slots. To analyze the performance of the learning algorithm, we introduce the notion of AoI regret, which is the difference between the expected cumulative AoI of the learning algorithm under consideration and the expected cumulative AoI of a genie algorithm that knows the reliability of the channels a priori. The AoI regret captures the penalty incurred by having to learn the statistics of the channels over the T time slots. The results are two-fold: first, we consider learning algorithms that employ wellknown solutions to the stochastic multi-armed bandit problem (such as -Greedy, Upper Confidence Bound, and Thompson Sampling) and show that their AoI regret scales as Θ(log T ); second, we develop a novel learning algorithm and show that it has O(1) regret. To the best of our knowledge, this is the first learning algorithm with bounded AoI regret.

We consider a single-hop wireless network with sources transmitting time-sensitive information to the destination over multiple unreliable channels. Packets from each source are generated according to a stochastic process with known statistics and the state of each wireless channel (ON/OFF) varies according to a stochastic process with unknown statistics. The reliability of the wireless channels is to be learned through observation. At every time-slot, the learning algorithm selects a single pair (source, channel) and the selected source attempts to transmit its packet via the selected channel. The probability of a successful transmission to the destination depends on the reliability of the selected channel. The goal of the learning algorithm is to minimize the Age-of-Information (AoI) in the network over T time-slots. To analyze its performance, we introduce the notion of AoI-regret, which is the difference between the expected cumulative AoI of the learning algorithm under consideration and the expected cumulative AoI of a genie algorithm that knows the reliability of the channels a priori. The AoI-regret captures the penalty incurred by having to learn the statistics of the channels over the T time-slots. The results are two-fold: first, we consider learning algorithms that employ well-known solutions to the stochastic multi-armed bandit problem (such as -Greedy, Upper Confidence Bound, and Thompson Sampling) and show that their AoI-regret scales as Θ(log T ); second, we develop a novel learning algorithm and show that it has O(1) regret. To the best of our knowledge, this is the first learning algorithm with bounded AoI-regret.