Optimizing age of information on real-life TCP/IP connections through reinforcement learning

Sert, Egemen; Sonmez, Canberk; Baghaee, Sajjad; Uysal-Biyikoğlu, Elif

doi:10.1109/siu.2018.8404794

Cited by 33 publications

(25 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, neither of the policies proposed in [47], [48] took into account the evolution of the battery level at the source and the variation of CSI over time in the process of decision-making. It is also worth noting that [22], [46], [49]- [52] have recently applied reinforcement learning-based algorithms to characterize the age-optimal policy. However, none of these works applied a DRL-based algorithm to efficiently design freshness-aware RF-powered communication systems.…”

Section: A Related Workmentioning

confidence: 99%

A Reinforcement Learning Framework for Optimizing Age of Information in RF-Powered Communication Systems

Abd-Elmagid

Dhillon

Παππάς

2020

IEEE Trans. Commun.

164

View full text Add to dashboard Cite

In this paper, we study a real-time monitoring system in which multiple source nodes are responsible for sending update packets to a common destination node in order to maintain the freshness of information at the destination. Since it may not always be feasible to replace or recharge batteries in all source nodes, we consider that the nodes are powered through wireless energy transfer (WET) by the destination. For this system setup, we investigate the optimal online sampling policy (referred to as the age-optimal policy) that jointly optimizes WET and scheduling of update packet transmissions with the objective of minimizing the long-term average weighted sum of Age of Information (AoI) values for different physical processes (observed by the source nodes) at the destination node, referred to as the sum-AoI. To solve this optimization problem, we first model this setup as an average cost Markov decision process (MDP) with finite state and action spaces. Due to the extreme curse of dimensionality in the state space of the formulated MDP, classical reinforcement learning algorithms are no longer applicable to our problem even for reasonable-scale settings. Motivated by this, we propose a deep reinforcement learning (DRL) algorithm that can learn the age-optimal policy in a computationally-efficient manner. We further characterize the structural properties of the age-optimal policy analytically, and demonstrate that it has a threshold-based structure with respect to the AoI values for different processes. We extend our analysis to characterize the structural properties of the policy that maximizes average throughput for our system setup, referred to as the throughputoptimal policy. Afterwards, we analytically demonstrate that the structures of the age-optimal and throughput-optimal policies are different. We also numerically demonstrate these structures as well as the impact of system design parameters on the optimal achievable average weighted sum-AoI.

show abstract

Section: A Related Workmentioning

confidence: 99%

A Reinforcement Learning Framework for Optimizing Age of Information in RF-Powered Communication Systems

Abd-Elmagid

Dhillon

Παππάς

2020

IEEE Trans. Commun.

164

View full text Add to dashboard Cite

show abstract

“…Ref. [60] introduced a deep reinforcement learning-based approach that can learn to minimize the AoI with no prior assumptions about network topology. Ref.…”

Section: Real-time Guaranteementioning

confidence: 99%

Ultra-reliable and low-latency communications: applications, opportunities and challenges

Feng

Lai

Luo

et al. 2021

Sci. China Inf. Sci.

View full text Add to dashboard Cite

In the upcoming 5G and beyond systems, ultra-reliable and low latency communication (URLLC) has been considered as the key enabler to support diverse mission-critical services, such as industrial automation, remote healthcare, and intelligent transportation. However, the two stringent requirements of URLLC: extremely low latency and ultra-strict reliability have posed great challenges in system designing. In this article, the basic concepts and the potential applications of URLLC are first introduced. Then, the state-ofthe-art research of URLLC in the physical layer, link layer and the network layer are overviewed. In addition, some potential research topics and challenges are also identified.

show abstract

“…However, depending on the application context, there is a wide range of objectives to be considered by system designers. There have been a number of works presenting how the concept of AoI and its relevant metrics can be incorporated in various areas, such as game theory [34,35] or reinforcement learning [36,37,38]. The number of domains in which AoI can be treated as a tool to facilitate the timely update of information in a system, is quite diverse.…”

Section: Motivationmentioning

confidence: 99%

Age of Information Aware Communication Systems : Modeling and Performance Analysis

Kosta

2020

Linköping Studies in Science and Technology. Dissertations

View full text Add to dashboard Cite

show abstract

Optimizing age of information on real-life TCP/IP connections through reinforcement learning

Cited by 33 publications

References 17 publications

A Reinforcement Learning Framework for Optimizing Age of Information in RF-Powered Communication Systems

A Reinforcement Learning Framework for Optimizing Age of Information in RF-Powered Communication Systems

Ultra-reliable and low-latency communications: applications, opportunities and challenges

Age of Information Aware Communication Systems : Modeling and Performance Analysis

Contact Info

Product

Resources

About