Long-Term CSI-Based Design for RIS-Aided Multiuser MISO Systems Exploiting Deep Reinforcement Learning

Ren, Hong; Pan, Cunhua; Wang, Liang; Kou, Zhoubing; Wang, Kezhi

doi:10.1109/lcomm.2021.3140155

Cited by 15 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…shows the convergence behavior of the proposed method. In order to better demonstrate the convergence performance of the proposed algorithm, instant reward and average reward are considered, among which the average reward is defined as follows [51]:…”

Section: B Convergencementioning

confidence: 99%

Deep Reinforcement Learning for RIS-Aided Multiuser Full-Duplex Secure Communications With Hardware Impairments

Peng

Zhang

Kong

et al. 2022

IEEE Internet Things J.

Self Cite

View full text Add to dashboard Cite

In this paper, we investigate a reconfigurable intelligent surface (RIS)-aided multiuser full-duplex secure communication system with hardware impairments at transceivers and RIS, where multiple eavesdroppers overhear the two-way transmitted signals simultaneously, and an RIS is applied to enhance the secrecy performance. Aiming at maximizing the sum secrecy rate (SSR), a joint optimization problem of the transmit beamforming at the base station (BS) and the reflecting beamforming at the RIS is formulated under the transmit power constraint of the BS and the unit modulus constraint of the phase shifters. As the environment is time-varying and the system is high-dimensional, this non-convex optimization problem is mathematically intractable. A deep reinforcement learning (DRL)-based algorithm is explored to obtain the satisfactory solution by repeatedly interacting with and learning from the dynamic environment. Extensive simulation results illustrate that the DRL-based secure beamforming algorithm is proved to be significantly effective in improving the SSR. It is also found that the performance of the DRL-based method can be greatly improved and the convergence speed of neural network can be accelerated with appropriate neural network parameters.

show abstract

Section: B Convergencementioning

confidence: 99%

Deep Reinforcement Learning for RIS-Aided Multiuser Full-Duplex Secure Communications With Hardware Impairments

Peng

Zhang

Kong

et al. 2022

IEEE Internet Things J.

Self Cite

View full text Add to dashboard Cite

show abstract

“…On the contrary, deep reinforcement learning (DRL) is a novel approach that combines deep learning (DL) and reinforcement learning (RL). It has been proven to be a significant breakthrough in non-convex optimization problems, including hybrid beamforming design [ 22 ], spectrum intelligence sensing [ 23 ], channel state estimation [ 24 ], and power allocation strategy optimization [ 25 ]. Compared with deep learning (DL), the DRL algorithm does not require a large amount of training labeled data as inputs and is therefore very friendly for the optimization of wireless communication systems, where obtaining data is more tedious.…”

Section: Introductionmentioning

confidence: 99%

Joint Beamforming Design for RIS-Assisted Integrated Satellite-HAP-Terrestrial Networks Using Deep Reinforcement Learning

Zhu

et al. 2023

Sensors

View full text Add to dashboard Cite

In this paper, we consider reconfigurable intelligent surface (RIS)-assisted integrated satellite high-altitude platform terrestrial networks (IS-HAP-TNs) that can improve network performance by exploiting the HAP stability and RIS reflection. Specifically, the reflector RIS is installed on the side of HAP to reflect signals from the multiple ground user equipment (UE) to the satellite. To aim at maximizing the system sum rate, we jointly optimize the transmit beamforming matrix at the ground UEs and RIS phase shift matrix. Due to the limitation of the unit modulus of the RIS reflective elements constraint, the combinatorial optimization problem is difficult to tackle effectively by traditional solving methods. Based on this, this paper studies the deep reinforcement learning (DRL) algorithm to achieve online decision making for this joint optimization problem. In addition, it is verified through simulation experiments that the proposed DRL algorithm outperforms the standard scheme in terms of system performance, execution time, and computing speed, making real-time decision making truly feasible.

show abstract

“…A dynamic power allocation problem with the time-varying channel is illustrated in [24] with a single transmit antenna, further studied in [25] by involving transmit beamforming in consideration and extended into multi-user scenario in [32]. Due to the appealing features of flexible deployment and sustainability in low power consumption, beamforming design of reconfigurable intelligent surface (RIS)-aided communications is proposed in [27]- [29], [34] to reduce computations compared with the alternating framework but requires unaffordable signaling overhead and complexity to obtain CSI. In terms of active beamforming using DRL, several efforts have been made on designing low complexity algorithms based on deep Q-network (DQN) [25], [31]- [33] and partially observed MDP [35] frameworks.…”

Section: Introductionmentioning

confidence: 99%

“…In terms of active beamforming using DRL, several efforts have been made on designing low complexity algorithms based on deep Q-network (DQN) [25], [31]- [33] and partially observed MDP [35] frameworks. However, existing works [24]- [34] assume that perfect CSIT or instantaneous channel gain via receiver feedback is known at the transmitter. Unfortunately, such an assumption is impractical in real-world systems with CSI feedback/acquisition delay and user mobility [6], [7].…”

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning for Multi-user Massive MIMO with Channel Aging

Feng¹,

Clerckx²

2023

Preprint

View full text Add to dashboard Cite

The design of beamforming for downlink multi-user massive multi-input multi-output (MIMO) relies on accurate downlink channel state information (CSI) at the transmitter (CSIT). In fact, it is difficult for the base station (BS) to obtain perfect CSIT due to user mobility, latency/feedback delay (between downlink data transmission and CSI acquisition). Hence, robust beamforming under imperfect CSIT is needed. In this paper, considering multiple antennas at all nodes (base station and user terminals), we develop a multi-agent deep reinforcement learning (DRL) framework for massive MIMO under imperfect CSIT, where the transmit and receive beamforming are jointly designed to maximize the average information rate of all users. Leveraging this DRL-based framework, interference management is explored and three DRL-based schemes, namely the distributed-learning-distributedprocessing scheme, partial-distributed-learning-distributed-processing, and central-learning-distributedprocessing scheme, are proposed and analyzed. This paper 1) highlights the fact that the DRL-based strategies outperform the random action-chosen strategy and the delay-sensitive strategy named as sample-and-hold (SAH) approach, and achieved over 90% of the information rate of two selected benchmarks with lower complexity: the zero-forcing channel-inversion (ZF-CI) with perfect CSIT and the Greedy Beam Selection strategy, 2) demonstrates the inherent robustness of the proposed designs in the presence of user mobility.

show abstract

Long-Term CSI-Based Design for RIS-Aided Multiuser MISO Systems Exploiting Deep Reinforcement Learning

Cited by 15 publications

References 18 publications

Deep Reinforcement Learning for RIS-Aided Multiuser Full-Duplex Secure Communications With Hardware Impairments

Deep Reinforcement Learning for RIS-Aided Multiuser Full-Duplex Secure Communications With Hardware Impairments

Joint Beamforming Design for RIS-Assisted Integrated Satellite-HAP-Terrestrial Networks Using Deep Reinforcement Learning

Deep Reinforcement Learning for Multi-user Massive MIMO with Channel Aging

Contact Info

Product

Resources

About