Deep Reinforcement Learning-Based Resource Allocation for Satellite Internet of Things with Diverse QoS Guarantee

Tang, Siqi; Pan, Zhisong; Hu, Guyu; Wu, Yang Chang; Li, Yunbo

doi:10.3390/s22082979

Cited by 13 publications

(8 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, the agent would be rewarded when this difference is increased, that is when throughput increases and energy consumption decreases. Authors in [105] and [106] also worked on energy efficiency optimization. Zhan et al in [105] modeled a joint design problem of mission completion time, UAV trajectory, as well as communication BS associations and solved it using multi-step DDQN RL algorithm to minimize the energy consumption of the UAV.…”

Section: ) Enhanced Qos/qoementioning

confidence: 99%

“…Zhan et al in [105] modeled a joint design problem of mission completion time, UAV trajectory, as well as communication BS associations and solved it using multi-step DDQN RL algorithm to minimize the energy consumption of the UAV. In [106] a deep reinforcement learning based online channel allocation and power control algorithm in a Satellite-IoT uplink scenario was proposed. The transmission channel and the power are determined by the intelligent agent based on contextual information.…”

Section: ) Enhanced Qos/qoementioning

confidence: 99%

See 1 more Smart Citation

Reinforcement Learning in the Sky: A Survey on Enabling Intelligence in NTN-Based Communications

et al. 2023

View full text Add to dashboard Cite

Non terrestrial networks (NTN) involving 'in the sky' objects such as low-earth orbit satellites, high altitude platform systems (HAPs) and Unmanned Aerial Vehicles (UAVs) are expected to be integral components of next generation cellular systems. With the deployment of 5G services and beyond, NTNs are leveraged to assist as aerial base stations in providing ubiquitous network connectivity and service to ground users or be deployed as aerial users connected to the cellular network. NTN-aided wireless communication offers multiple benefits such as mobility, flexibility, resistance to ground physical attacks and wide coverage. However, due to their limited resources and the current design of terrestrial cellular systems that do not account for aerial users, and other restrictions such as service requirements, limited available power and storage resources on high-throughput satellites, resource allocation, location of the high altitude platform base station and the flight trajectory of the UAVs need to be intelligently controlled to satisfy various objectives both from an aerial base station and overall network perspectives. To achieve this, many works have explored Reinforcement Learning (RL) techniques to allow aerial platforms in non-terrestrial networks to learn from past observations and achieve some optimal control policy. In this paper and differently from prior surveys, we contribute a comprehensive review of the control objectives required by non-terrestrial platforms that have been solved using RL formulations. We provide an up-to-date overview of the latest applications of RL techniques for different NTN-aided wireless communication aspects. The survey focuses on Markov Decision Process (MDP) formulations in terms of states, actions, and rewards. We synthesize a taxonomy from the surveyed literature and provide a comprehensive representation of the current usages of RL in NTN-aided wireless communications. A qualitative analysis of the level of realism achieved in the works presented in the literature is provided based on several factors that pertain to the simulation environment, station deployment setting, wireless channel assumption, and energy considerations. We also curate a list of challenges that remain to be considered by the research community in order to achieve more efficient deployments and close the simulation-to-reality gap.

show abstract

Section: ) Enhanced Qos/qoementioning

confidence: 99%

Section: ) Enhanced Qos/qoementioning

confidence: 99%

Reinforcement Learning in the Sky: A Survey on Enabling Intelligence in NTN-Based Communications

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Using DL, flexible payload optimization is able to reduce this unmet capacity by 32%. 32 Authors of Tang et al 118 propose a deep reinforcement learning system for resource allocation in satellite IoT. They use an intelligent agent for transmission channel determination and power usage on the information available.…”

Section: Resource Allocation and Managementmentioning

confidence: 99%

Machine Learning and Deep Learning powered satellite communications: Enabling technologies, applications, open challenges, and future research directions

Bhattacharyya

Nambiar

Ojha

et al. 2023

Satell Commun Network

View full text Add to dashboard Cite

SummaryThe recent wave of creating an interconnected world through satellites has renewed interest in satellite communications. Private and government‐funded space agencies are making advancements in the creation of satellite constellations, and the introduction of 5G has brought a new focus to a fully connected world. Satellites are the proposed solutions for establishing high throughput and low latency links to remote, hard‐to‐reach areas. This has caused the injection of many satellites in Earth's orbit, which has caused many discrepancies. There is a need to establish highly adaptive and flexible satellite systems to overcome this. Machine Learning (ML) and Deep Learning (DL) have gained much popularity when it comes to communication systems. This review extensively provides insight into ML and DL's utilization in satellite communications. This review covers how satellite communication subsystems and other satellite system applications can be implemented through Artificial Intelligence (AI) and the ongoing open challenges and future directions.

show abstract

“…They exploit Upper Confidence Bound (UCB) algorithms for frequency allocation, which are based on simple computation of indexes and known to achieve the optimal performance asymptotically [7]- [10]. There are also studies [11], [12] that address the time-varying interference problems in the satellite network using Deep Reinforcement Learning (DRL) techniques. By training neural network models, the authors find the best beam pattern and bandwidth allocation.…”

Section: Introductionmentioning

confidence: 99%

Dynamic Downlink Interference Management in LEO Satellite Networks Without Direct Communications

Yun

et al. 2023

IEEE Access

View full text Add to dashboard Cite

We investigate effective interference management for Low Earth Orbit (LEO) satellite networks that provide downlink services to ground users and share the same frequency spectrum range. Since there are multi-group LEO satellites with different constellation orbits, the ground users will experience time-varying interference due to the overlapping of main/side lobes of the satellite beams, which becomes even challenging when the interfering satellites cannot communicate directly. To address the problem, we consider two LEO satellite groups that provide communication service in the same ground area, while competing for communication resources. We develop solutions that maximize the throughput and manage the time-varying interference under a certain level, without explicit message exchanges between the satellite groups. By exploiting statistical learning and deep reinforcement learning techniques, we develop learning-based resource allocation schemes and evaluate their performance through extensive simulations. We show their effectiveness under different reward settings and different interference managements, and demonstrate that a Deep Q-Network (DQN)-based scheme can achieve the close-to-optimal performance.

show abstract

Deep Reinforcement Learning-Based Resource Allocation for Satellite Internet of Things with Diverse QoS Guarantee

Cited by 13 publications

References 25 publications

Reinforcement Learning in the Sky: A Survey on Enabling Intelligence in NTN-Based Communications

Reinforcement Learning in the Sky: A Survey on Enabling Intelligence in NTN-Based Communications

Machine Learning and Deep Learning powered satellite communications: Enabling technologies, applications, open challenges, and future research directions

Dynamic Downlink Interference Management in LEO Satellite Networks Without Direct Communications

Contact Info

Product

Resources

About