Intelligent Action Selection for NGSO Networks With Interference Constraints: A Modified Q-Learning Approach

Ren, Zixuan; Jin, Jin; Li, Wei; Zhan, Yafeng

doi:10.1109/taes.2021.3130832

Cited by 11 publications

(8 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For limited power resources, dynamic power allocation is more suitable for accommodating varying traffic demands than fixed power allocation in practical systems. Moreover, dynamic power allocation is also an effective way to mitigate interference, and it has been applied to diverse scenarios, such as GEO and LEO coexistence scenario 15,16 and LEO and terrestrial network coexistence scenario 17 . For the GEO and LEO coexistence scenario, the guiding principle of LEO satellite deployment is that the LEO system cannot cause unacceptable interference to the GEO system.…”

Section: Related Workmentioning

confidence: 99%

“…So, the interference caused by LEO satellites should be mitigated by certain measures to protect GEO users. However, under the constraints of GEO network interference protection, it is difficult to maximize LEO system performance 17 . To ensure the service quality of GEO satellite users and maximize the throughput for LEO users simultaneously, the adaptive beam power control is proposed, 18,19 which can improve spectrum efficiency and ensure user fairness.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Joint beam power and pointing management in multi‐beam low earth orbit and low earth orbit co‐existing satellite system

He,

Cui,

et al. 2024

Satell Commun Network

View full text Add to dashboard Cite

SummaryRecently, the low earth orbit (LEO) satellite has attracted much attention due to its advantages of low latency and construction costs. However, the scarcity of frequency spectrum resources has resulted in these satellites sharing the same spectrum to provide services for ground users. So, challenges such as overlapping beam coverage and inter‐beam interference occur in the multi‐beam LEO satellite coexistence scenario. Therefore, this paper focuses on the issue of interference coexistence in LEO and LEO (LEO–LEO) co‐existing scenario, where the initially established system is the primary system, and the later established system is the secondary system. The interference coexistence problem is modeled as a multi‐objective optimization problem. The joint beam power and pointing management (JBPPM) based on particle swarm optimization (PSO) is proposed. The power control is applied to the LEO satellites in the secondary system to reduce the interference to the primary system user. Meanwhile, beam pointing optimization is adopted by the LEO users in the secondary system to avoid interference from the primary system and ensure the minimum communication ability. The simulation results verify that the proposed technique can adapt to various coexistence conditions and outperform the existing method.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Joint beam power and pointing management in multi‐beam low earth orbit and low earth orbit co‐existing satellite system

He,

Cui,

et al. 2024

Satell Commun Network

View full text Add to dashboard Cite

show abstract

“…It means that frequency resources, in the physical form of communication beams (comm‐beams), are used statically. However, for NGSO networks, the distribution of comm‐beams carrying different frequencies is complex and changes dynamically in real time, due to the large scale, dynamic and multinode features 7,8 . As shown in Figure 1, the frequency subbands

f_{1}, f_{2}, f_{3}

, and

f_{4}

occur in different spatial locations in the physical form of comm‐beams in downlink case, and the ESs include in‐motion ESs, gateway stations, user terminals, and so on.…”

Section: Introductionmentioning

confidence: 99%

Mapping spatial distribution of comm‐satellite's beam based on ground omni‐antennas

Ren

Jin

et al. 2023

Satell Commun Network

View full text Add to dashboard Cite

Summary The current satellite communications (SatComs) systems are composed of a large number of satellites, beams and terrestrial devices. Due to their multinode dynamic nature, the usage of frequency resources is variable, complex and difficult to characterize. In particular, with the development of satellite‐borne phased array antenna technology, SatCom beams carrying different frequencies are directionally and dynamically distributed in global scale. Mapping and locating the spatial beam distributions of communication satellite (comm‐satellite) are the bases of intersystem cofrequency interference mitigation and spatial frequency reuse. In this paper, we design a data selection–multiparameter fitting iteration (DS‐MFI) algorithm on the basis of ground‐based omnidirectional antennas. The proposed approach can effectively map the spatial distribution of comm‐satellite's beam, including satellite transmitter position, equal‐gain off‐axis angle, and beam pointing in azimuth and elevation. Simulation results verify the effectiveness of the proposed approach for satellites with fixed or steerable beams at different altitudes. Furthermore, the results become increasingly accurate as the dense of ground omni‐antenna increases.

show abstract

“…(2020) applied Q-learning to the power allocation problem in satellite-to-ground communication using LEO satellites; Zhou et al . (2021) proposed an adaptive routing strategy based on the double Q-learning for Satellite Internet of Things (S-IoT); Ren et al . (2022) proposed a modified Q-Learning approach based on an iterative greedy algorithm to upgrade the system performance of non-geostationary orbit network.…”

Section: Introductionmentioning

confidence: 99%

“…It has been implemented in various areas. Take the field of satellite communications as an example, Jiang and Zhu (2020) proposed a long-term optimal capacity allocation algorithm to optimize the long-term utility of the multi-layer satellite networks based on Q-learning; Tsuchida et al (2020) applied Q-learning to the power allocation problem in satellite-to-ground communication using LEO satellites; Zhou et al (2021) proposed an adaptive routing strategy based on the double Qlearning for Satellite Internet of Things (S-IoT); Ren et al (2022) proposed a modified Q-Learning approach based on an iterative greedy algorithm to upgrade the system performance of nongeostationary orbit network. The reward function of Q-Learning guides the agent and determines the performance of algorithm to some extent.…”

mentioning

confidence: 99%

GERT-Q-learning model for intelligent QoS dynamic optimization of Inmarsat STN based on grey clustering of delay and delay variation

Hua

Fang

Zhang

et al. 2023

View full text Add to dashboard Cite

PurposeThis paper aims to implement quality of service(QoS) dynamic optimization for the integrated satellite-terrestrial network(STN) of the fifth-generation Inmarsat system(Inmarsat-5).Design/methodology/approachThe structure and operational logic of Inmarsat-5 STN are introduced to build the graphic evaluation and review technique(GERT) model. Thus, the equivalent network QoS metrics can be derived from the analytical algorithm of GERT. The center–point mixed possibility functions of average delay and delay variation are constructed considering users' experiences. Then, the grey clustering evaluation of link QoS is obtained combined with the two-stage decision model to give suitable rewards for the agent of GERT-Q-learning, which realizes the intelligent optimization mechanism under real-time monitoring data.FindingsA case study based on five time periods of monitoring data verifies the adaptability of the proposed method. On the one hand, grey clustering based on possibility function enables a more effective measurement of link QoS from the users' perspective. On the other hand, the method comparison intuitively shows that the proposed method performs better.Originality/valueWith the development trend of integrated communication, STN has become an important research object in satellite communications. This paper establishes a modular and extensible optimization framework whose loose coupling structure and flexibility facilitate management and development. The grey-clustering-based GERT-Q-Learning model has the potential to maximize design and application benefits of STN throughout its life cycle.

show abstract

Intelligent Action Selection for NGSO Networks With Interference Constraints: A Modified Q-Learning Approach

Cited by 11 publications

References 35 publications

Joint beam power and pointing management in multi‐beam low earth orbit and low earth orbit co‐existing satellite system

Joint beam power and pointing management in multi‐beam low earth orbit and low earth orbit co‐existing satellite system

Mapping spatial distribution of comm‐satellite's beam based on ground omni‐antennas

GERT-Q-learning model for intelligent QoS dynamic optimization of Inmarsat STN based on grey clustering of delay and delay variation

Contact Info

Product

Resources

About