A Reinforcement Learning Approach for an IRS-assisted NOMA Network

Xie, Ximing; Jiao, Shiyu; Ding, Zhiguo

doi:10.48550/arxiv.2106.09611

Cited by 2 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Any negative reward will work as the agent will try to avoid such action in the future. We will use the sum of the rate deficit across all users as the negative reward [18]. The set ε contains users j = 1, .., J, ε ∈ N whose QoS are not satisfied at time-step t. Thus, we define the sum of the rate deficit across all users as…”

Section: Robust Design Problem As Td3 Environmentmentioning

confidence: 99%

“…In [17], Meng et al proposed a DRL-based solution for sum-rate maximization in multi-cell networks. Xiao et al proposed a deep deterministic policy gradient (DDPG) based solution to jointly optimize the beamforming and phase shifts of IRS elements for sum-rate maximization in an IRS-assisted MISO-NOMA system [18]. In [19], Gao et al proposed a deep Q-network (DQN) based algorithm to jointly optimize IRS phase shifts and cluster power allocation in a NOMA system using the zero forcing approach.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning-Based Robust Design for an IRS-Assisted MISO-NOMA System

Waraiet,

Cumanan,

Ding

et al. 2024

Trans. Mach. Learn. Comm. Netw.

View full text Add to dashboard Cite

In this paper, we propose a robust design for an intelligent reflecting surface (IRS)-assisted multiple-input single output non-orthogonal multiple access (NOMA) system. By considering channel uncertainties, the original robust design problem is formulated as a sum-rate maximization problem under a set of constraints. In particular, the uncertainties associated with reflected channels through IRS elements and direct channels are taken into account in the design and they are modelled as bounded errors. However, the original robust problem is not jointly convex in terms of beamformers at the base station and phase shifts of IRS elements. Therefore, we reformulate the original robust design as a reinforcement learning problem and develop an algorithm based on the twin-delayed deep deterministic policy gradient agent (also known as TD3). In particular, the proposed algorithm solves the original problem by jointly designing the beamformers and the phase shifts, which is not possible with conventional optimization techniques. Numerical results are provided to validate the effectiveness and evaluate the performance of the proposed robust design. In particular, the results demonstrate the competitive and promising capabilities of the proposed robust algorithm, which achieves significant gains in terms of robustness and system sum-rates over the baseline deep deterministic policy gradient agent. In addition, the algorithm has the ability to deal with fixed and dynamic channels, which gives deep reinforcement learning methods an edge over hand-crafted convex optimization-based algorithms.INDEX TERMS MISO-NOMA, power allocation, non-convex optimization, reinforcement learning, robust design.

show abstract

Section: Robust Design Problem As Td3 Environmentmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning-Based Robust Design for an IRS-Assisted MISO-NOMA System

Waraiet,

Cumanan,

Ding

et al. 2024

Trans. Mach. Learn. Comm. Netw.

View full text Add to dashboard Cite

show abstract

“…The work in [22] adopted the zero-forcing beamforming (ZFBF) technique while utilizing a deep Q-network (DQN) agent for optimizing phase shifts of the IRS elements. Xie et al used DDPG to jointly optimize the beamforming vectors and IRS phase shifts for the sum-rate maximization problem [23]. The work in [24] proposed a multi-agent DRL-based design that jointly optimizes the subcarrier assignment, power allocation, and IRS phase shifts in NOMA-assisted semi-grant-free systems, while the resource allocation problem for NOMAunmanned aerial vehicle system was considered in [25].…”

Section: Introductionmentioning

confidence: 99%

Outage-Constrained Robust Resource Allocation Framework for IRS-Empowered NOMA Systems: A DRL-Based Joint Design

Waraiet,

Cumanan

2024

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

In this paper, we propose a robust resource allocation framework for an intelligent reflecting surface (IRS)-assisted multiple-input single-output (MISO) non-orthogonal multiple access (NOMA) system. In particular, a long-term robust sum-rate maximization problem is considered. The impacts of imperfect channel estimation on both the transmitter and the receiver are taken into account with an outage-constrained robust design approach. More specifically, the statistical error model is used to model the unbounded channel uncertainty in the system. However, the joint robust resource allocation problem is a mixed-integer optimization problem, which cannot be solved directly using conventional optimization algorithms. A correlation-based user pairing algorithm is proposed to group the users into clusters. Furthermore, the resource allocation problem with clustered users is reformulated as a reinforcement learning environment. Subsequently, a twin-delayed deep deterministic policy gradient (TD3) agent is developed to solve the outage-constrained robust resource allocation problem. Extensive simulation results are provided to demonstrate the superior performance of the developed TD3 agent over existing algorithms in the literature.

show abstract

A Reinforcement Learning Approach for an IRS-assisted NOMA Network

Cited by 2 publications

References 10 publications

Deep Reinforcement Learning-Based Robust Design for an IRS-Assisted MISO-NOMA System

Deep Reinforcement Learning-Based Robust Design for an IRS-Assisted MISO-NOMA System

Outage-Constrained Robust Resource Allocation Framework for IRS-Empowered NOMA Systems: A DRL-Based Joint Design

Contact Info

Product

Resources

About