Joint Coordinated Beamforming and Power Splitting Ratio Optimization in MU-MISO SWIPT-Enabled HetNets: A Multi-Agent DDQN-Based Approach

Zhang, Ruichen; Xiong, Ke; Lu, Yang; Gao, Bo; Fan, Pingyi; Letaief, Khaled B.

doi:10.1109/jsac.2021.3118397

Cited by 60 publications

(16 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where c t and c r denote the codewords of two codebooks and the cardinal number of action space A is S t × S r . The design of codebooks comes from [42] which is also applied in [25], [31], [32], and introduced here as a quantization of beam directions. To specify each element, we define matrix…”

Section: ) Downlink Training and Uplink Feedbackmentioning

confidence: 99%

“…Due to the appealing features of flexible deployment and sustainability in low power consumption, beamforming design of reconfigurable intelligent surface (RIS)-aided communications is proposed in [27]- [29], [34] to reduce computations compared with the alternating framework but requires unaffordable signaling overhead and complexity to obtain CSI. In terms of active beamforming using DRL, several efforts have been made on designing low complexity algorithms based on deep Q-network (DQN) [25], [31]- [33] and partially observed MDP [35] frameworks. However, existing works [24]- [34] assume that perfect CSIT or instantaneous channel gain via receiver feedback is known at the transmitter.…”

Section: Introductionmentioning

confidence: 99%

“…To our best knowledge, predicting the beamformers of both transmitter and receiver with imperfect CSIT is never considered in DRL-based papers. Instead, all the existing work focuses on high-level multi-cell single-user (SU) single-input-single-output (SISO) [24], [26] and multi-input-single-output (MISO) [25], [28], [31], [35] scenarios without considering multiple receive antenna cases, which motivates this work. In addition, compared with MISO [24] and SISO [25] scenarios, mitigating the interference of the massive MIMO scenarios is more challenging since we not only need to alleviate the multi-user interference but also need to jointly design the receive beamforming to combat the inter-stream interference.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep Reinforcement Learning for Multi-user Massive MIMO with Channel Aging

Feng¹,

Clerckx²

2023

Preprint

View full text Add to dashboard Cite

The design of beamforming for downlink multi-user massive multi-input multi-output (MIMO) relies on accurate downlink channel state information (CSI) at the transmitter (CSIT). In fact, it is difficult for the base station (BS) to obtain perfect CSIT due to user mobility, latency/feedback delay (between downlink data transmission and CSI acquisition). Hence, robust beamforming under imperfect CSIT is needed. In this paper, considering multiple antennas at all nodes (base station and user terminals), we develop a multi-agent deep reinforcement learning (DRL) framework for massive MIMO under imperfect CSIT, where the transmit and receive beamforming are jointly designed to maximize the average information rate of all users. Leveraging this DRL-based framework, interference management is explored and three DRL-based schemes, namely the distributed-learning-distributedprocessing scheme, partial-distributed-learning-distributed-processing, and central-learning-distributedprocessing scheme, are proposed and analyzed. This paper 1) highlights the fact that the DRL-based strategies outperform the random action-chosen strategy and the delay-sensitive strategy named as sample-and-hold (SAH) approach, and achieved over 90% of the information rate of two selected benchmarks with lower complexity: the zero-forcing channel-inversion (ZF-CI) with perfect CSIT and the Greedy Beam Selection strategy, 2) demonstrates the inherent robustness of the proposed designs in the presence of user mobility.

show abstract

Section: ) Downlink Training and Uplink Feedbackmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning for Multi-user Massive MIMO with Channel Aging

Feng¹,

Clerckx²

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Despite the different design aim, our work instead develops a game-based interactive method additionally controlled by a threshold to meet our time constraint. In [ 45 ], a sum rate maximization problem was formulated for SWIPT enabled HetNets, which jointly optimizes transmit beamforming vectors and power splitting ratios. With the multi-agent DDPG method for the user equipment (UE) without mobility, this work exhibits a notable performance gain when compared with the fixed beamforming design.…”

Section: Introductionmentioning

confidence: 99%

Joint Beamforming, Power Allocation, and Splitting Control for SWIPT-Enabled IoT Networks with Deep Reinforcement Learning and Game Theory

Liu

Lin

et al. 2022

Sensors

View full text Add to dashboard Cite

Future wireless networks promise immense increases on data rate and energy efficiency while overcoming the difficulties of charging the wireless stations or devices in the Internet of Things (IoT) with the capability of simultaneous wireless information and power transfer (SWIPT). For such networks, jointly optimizing beamforming, power control, and energy harvesting to enhance the communication performance from the base stations (BSs) (or access points (APs)) to the mobile nodes (MNs) served would be a real challenge. In this work, we formulate the joint optimization as a mixed integer nonlinear programming (MINLP) problem, which can be also realized as a complex multiple resource allocation (MRA) optimization problem subject to different allocation constraints. By means of deep reinforcement learning to estimate future rewards of actions based on the reported information from the users served by the networks, we introduce single-layer MRA algorithms based on deep Q-learning (DQN) and deep deterministic policy gradient (DDPG), respectively, as the basis for the downlink wireless transmissions. Moreover, by incorporating the capability of data-driven DQN technique and the strength of noncooperative game theory model, we propose a two-layer iterative approach to resolve the NP-hard MRA problem, which can further improve the communication performance in terms of data rate, energy harvesting, and power consumption. For the two-layer approach, we also introduce a pricing strategy for BSs or APs to determine their power costs on the basis of social utility maximization to control the transmit power. Finally, with the simulated environment based on realistic wireless networks, our numerical results show that the two-layer MRA algorithm proposed can achieve up to 2.3 times higher value than the single-layer counterparts which represent the data-driven deep reinforcement learning-based algorithms extended to resolve the problem, in terms of the utilities designed to reflect the trade-off among the performance metrics considered.

show abstract

“…There has been a growing interest in applying machine learning techniques in wireless system design [23]- [28]. Particularly, the work in these literatures used DRL approach to train some optimal strategies, e.g.…”

Section: Introductionmentioning

confidence: 99%

Energy Consumption Minimization for Data Collection From Wirelessly-Powered IoT Sensors: Session-Specific Optimal Design With DRL

Yang

Alouini

2022

IEEE Sensors J.

View full text Add to dashboard Cite

Reliable and energy-efficient data collection from resource-limited sensors is essential to the success of future Internet of Things (IoT). In this article, we study the energy consumption minimization problem during the data collection from a generic wirelessly-powered sensor. Specifically, we determine the optimal data collection parameters, in terms of charging duration and charging power as well as sensor transmission rate, in real time according to the instantaneous channel condition while satisfying a certain latency constraint. For the scenario of ideal rate adaptive transmission with linear energy harvesting, we derive closed-form expressions for optimal transmission parameters. We also establish the condition on channel quality for successful data collection under a latency constraint. For the more practical case of finite block-length transmission with non-linear energy harvesting, we develop a deep reinforcement learning (DRL) solution for efficient online implementation. We also propose an online tuning scheme to cater for model inaccuracy and environment variation. The accuracy and effectiveness of our proposed approaches are verified by comparing with benchmark schemes. Our DRL-based approach has broad applicability and can solve other real-time optimal design problems in wireless communications.

show abstract

Joint Coordinated Beamforming and Power Splitting Ratio Optimization in MU-MISO SWIPT-Enabled HetNets: A Multi-Agent DDQN-Based Approach

Cited by 60 publications

References 53 publications

Deep Reinforcement Learning for Multi-user Massive MIMO with Channel Aging

Deep Reinforcement Learning for Multi-user Massive MIMO with Channel Aging

Joint Beamforming, Power Allocation, and Splitting Control for SWIPT-Enabled IoT Networks with Deep Reinforcement Learning and Game Theory

Energy Consumption Minimization for Data Collection From Wirelessly-Powered IoT Sensors: Session-Specific Optimal Design With DRL

Contact Info

Product

Resources

About