Adaptive Routing Strategy Based on Improved Q-learning for Satellite Internet of Things

Gong, Xiaotian; Sun, Lijuan; Zhou, Jian; Wang, Juan; Xiao, Fu

doi:10.1007/978-3-030-68884-4_13

Cited by 3 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The scarce literature on beam and gateway routing mainly focuses on how to route users to gateways in order to minimize delay. While most studies suggest non-AI load balancing solutions (Torkzaban et al 2020;Yang et al 2016), some authors have proposed GA (Rao and Wang 2011) and Q-Learning (Gong et al 2020) frameworks in the context of highly-varying environments.…”

Section: Ai and Learning-based Methods For Resource Allocation Subpro...mentioning

confidence: 99%

“…Wang 2011; Tirmizi, Mishra, and Zadgaonkar 2015; Durand and Abrão 2017), and show promising results when dealing with multiple subproblems (Camino et al 2014;Pachler et al 2020;He, Jia, and Zhong 2017). In cases where the dimensionality is so high that these techniques are no longer practical, learning-based alternatives have been explored (Hu et al 2020b;Gong et al 2020;Wang et al 2020a;Luis et al 2019).…”

Section: Ai-based Decision-making In the Subsystemsmentioning

confidence: 99%

“…The goal of the RTE is not to allocate on all possible variables, but rather to a lower-dimensional subset that does not compromise a fast convergence time. In that sense, the Reinforcement Learning-based agents presented in the literature could be better suited to reallocate power (Zhang et al 2020;Liao et al 2020;Ferreira et al 2018;Luis et al 2019Luis et al , 2020, reassign frequencies (Liao et al 2020;Zheng et al 2020;Ferreira et al 2018;Hu et al 2018), reroute (Gong et al 2020), or rearrange the beam placement (Hu et al 2020b). In the case of routing and beam placement tasks, the lack of more studies also opens the door to consider metaheuristic-based alternatives, such as (Wang et al 2020a;Deng et al 2018;He, Jia, and Zhong 2017;Anzalchi et al 2010;Angeletti, Fernandez Prim, and Rinaldo 2006;Rao and Wang 2011).…”

Section: Ai-based Decision-making In the Subsystemsmentioning

confidence: 99%

See 2 more Smart Citations

Towards Autonomous Satellite Communications: An AI-based Framework to Address System-level Challenges

Garau-Luis¹,

Eiskowitz²,

Pachler³

et al. 2021

Preprint

View full text Add to dashboard Cite

The next generation of satellite constellations is designed to better address the future needs of our connected society: highly-variable data demand, mobile connectivity, and reaching more under-served regions. Artificial Intelligence (AI) and learning-based methods are expected to become key players in the industry, given the poor scalability and slow reaction time of current resource allocation mechanisms. While AI frameworks have been validated for isolated communication tasks or subproblems, there is still not a clear path to achieve fully-autonomous satellite systems. Part of this issue results from the focus on subproblems when designing models, instead of the necessary system-level perspective. In this paper we try to bridge this gap by characterizing the systemlevel needs that must be met to increase satellite autonomy, and introduce three AI-based components (Demand Estimator, Offline Planner, and Real Time Engine) that jointly address them. We first do a broad literature review on the different subproblems and identify the missing links to the systemlevel goals. In response to these gaps, we outline the three necessary components and highlight their interactions. We also discuss how current models can be incorporated into the framework and possible directions of future work.

show abstract