LEO Satellite Channel Allocation Scheme Based on Reinforcement Learning

Zheng, Fei; Pi, Zhao; Zhou, Zou; Wang, Kaixuan

doi:10.1155/2020/8868888

Cited by 21 publications

(16 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Both metaheuristic (GA, (He, Jia, and Zhong 2017;Tirmizi, Mishra, and Zadgaonkar 2015), SA, (Vidal, Legay, and Goussetis 2020)) and RL-based (Q-learning, (Hu et al 2018)) formulations have proven successful in solving this task. Some studies do consider flexible bandwidth and address it with both metaheuristic frameworks (GA, (Angeletti, Fernandez Prim, and Rinaldo 2006;Paris et al 2019), SA (Cocco et al 2018)) and learning-based models (Q-Learning, (Liao et al 2020;Zheng et al 2020), supervised learning with neural networks, (Funabiki and Nishikawa 1997;Salcedo-Sanz and Bousoño-Calzón 2005)). While metaheuristics are better suited for scenarios with fewer beams, the highdimensionality of real operations makes learning-based techniques a competitive solution for time-constrained contexts.…”

Section: Ai and Learning-based Methods For Resource Allocation Subpro...mentioning

confidence: 99%

Towards Autonomous Satellite Communications: An AI-based Framework to Address System-level Challenges

Garau-Luis¹,

Eiskowitz²,

Pachler³

et al. 2021

Preprint

View full text Add to dashboard Cite

The next generation of satellite constellations is designed to better address the future needs of our connected society: highly-variable data demand, mobile connectivity, and reaching more under-served regions. Artificial Intelligence (AI) and learning-based methods are expected to become key players in the industry, given the poor scalability and slow reaction time of current resource allocation mechanisms. While AI frameworks have been validated for isolated communication tasks or subproblems, there is still not a clear path to achieve fully-autonomous satellite systems. Part of this issue results from the focus on subproblems when designing models, instead of the necessary system-level perspective. In this paper we try to bridge this gap by characterizing the systemlevel needs that must be met to increase satellite autonomy, and introduce three AI-based components (Demand Estimator, Offline Planner, and Real Time Engine) that jointly address them. We first do a broad literature review on the different subproblems and identify the missing links to the systemlevel goals. In response to these gaps, we outline the three necessary components and highlight their interactions. We also discuss how current models can be incorporated into the framework and possible directions of future work.

show abstract

Section: Ai and Learning-based Methods For Resource Allocation Subpro...mentioning

confidence: 99%

Towards Autonomous Satellite Communications: An AI-based Framework to Address System-level Challenges

Garau-Luis¹,

Eiskowitz²,

Pachler³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…An interesting work is reported by Zheng et al [158] in which the authors propose a single-agent Q-learning-based RL model to address the problem of combination allocation of fixed channel pre-allocation and dynamic channel scheduling in a network architecture of LEO satellites that utilizes a centralized resource pool. In their model, the satellite serves as an agent whose action is discrete, corresponding to assigning channels to users.…”

Section: ) In Satellite Networkmentioning

confidence: 99%

The Frontiers of Deep Reinforcement Learning for Resource Management in Future Wireless HetNets: Techniques, Challenges, and Research Directions

Alwarafy

Abdallah

Çiftler

et al. 2022

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

Next generation wireless networks are expected to be extremely complex due to their massive heterogeneity in terms of the types of network architectures they incorporate, the types and numbers of smart IoT devices they serve, and the types of emerging applications they support. In such large-scale and heterogeneous networks (HetNets), radio resource allocation and management (RRAM) becomes one of the major challenges encountered during system design and deployment. In this context, emerging Deep Reinforcement Learning (DRL) techniques are expected to be one of the main enabling technologies to address the RRAM in future wireless HetNets. In this paper, we conduct a systematic in-depth, and comprehensive survey of the applications of DRL techniques in RRAM for next generation wireless networks. Towards this, we first overview the existing traditional RRAM methods and identify their limitations that motivate the use of DRL techniques in RRAM. Then, we provide a comprehensive review of the most widely used DRL algorithms to address RRAM problems, including the value-and policy-based algorithms. The advantages, limitations, and use-cases for each algorithm are provided. We then conduct a comprehensive and in-depth literature review and classify existing related works based on both the radio resources they are addressing and the type of wireless networks they are investigating. To this end, we carefully identify the types of DRL algorithms utilized in each related work, the elements of these algorithms, and the main findings of each related work. Finally, we highlight important open challenges and provide insights into several future research directions in the context of DRL-based RRAM. This survey is intentionally designed to guide and stimulate more research endeavors towards building efficient and fine-grained DRL-based RRAM schemes for future wireless networks.

show abstract

“…Zheng et al [149] propose a single-agent Q-learning-based RL model to address the problem of combination allocation of fixed channel pre-allocation and dynamic channel scheduling in a network architecture of LEO satellites that utilizes a centralized resource pool. In their model, the satellite serves as an agent whose action is discrete, corresponding to assigning channels to users.…”

Section: ) In Iot and Other Emerging Wireless Networkmentioning

confidence: 99%

Deep Reinforcement Learning for Radio Resource Allocation and Management in Next Generation Heterogeneous Wireless Networks: A Survey

Alwarafy¹,

Abdallah²,

Çiftler³

et al. 2021

Preprint

View full text Add to dashboard Cite

<div>Next generation wireless networks are expected to be extremely complex due to their massive heterogeneity in terms of the types of network architectures they incorporate, the types and numbers of smart IoT devices they serve, and the types of emerging applications they support. In such large-scale and heterogeneous networks (HetNets), radio resource allocation and management (RRAM) becomes one of the major challenges encountered during system design and deployment. In this context, emerging Deep Reinforcement Learning (DRL) techniques are expected to be one of the main enabling technologies to address the RRAM in future wireless HetNets. In this paper, we conduct a systematic in-depth, and comprehensive survey of the applications of DRL techniques in RRAM for next generation wireless networks. Towards this, we first overview the existing traditional RRAM methods and identify their limitations that motivate the use of DRL techniques in RRAM. Then, we provide a comprehensive review of the most widely used DRL algorithms to address RRAM problems, including the value- and policy-based algorithms. The advantages, limitations, and use-cases for each algorithm are provided. We then conduct a comprehensive and in-depth literature review and classify existing related works based on both the radio resources they are addressing and the type of wireless networks they are investigating. To this end, we carefully identify the types of DRL algorithms utilized in each related work, the elements of these algorithms, and the main findings of each related work. Finally, we highlight important open challenges and provide insights into several future research directions in the context of DRL-based RRAM. This survey is intentionally designed to guide and stimulate more research endeavors towards building efficient and fine-grained DRL-based RRAM schemes for future wireless networks.</div>

show abstract

LEO Satellite Channel Allocation Scheme Based on Reinforcement Learning

Cited by 21 publications

References 12 publications

Towards Autonomous Satellite Communications: An AI-based Framework to Address System-level Challenges

Towards Autonomous Satellite Communications: An AI-based Framework to Address System-level Challenges

The Frontiers of Deep Reinforcement Learning for Resource Management in Future Wireless HetNets: Techniques, Challenges, and Research Directions

Deep Reinforcement Learning for Radio Resource Allocation and Management in Next Generation Heterogeneous Wireless Networks: A Survey

Contact Info

Product

Resources

About