Deep Reinforcement Learning-based Power Allocation in Uplink Cell-Free Massive MIMO

Abstract

A cell-free massive multiple-input multiple-output (MIMO) uplink is investigated in this paper. We address a power allocation design problem that considers two conflicting metrics, namely the sum rate and fairness. Different weights are allocated to the sum rate and fairness of the system, based on the requirements of the mobile operator. The knowledge of the channel statistics is exploited to optimize power allocation. We propose to employ large scale-fading (LSF) coefficients as the input of a twin delayed deep deterministic policy gradient (TD3). This enables us to solve the non-convex sum rate fairness trade-off optimization problem efficiently. Then, we exploit a use-and-then-forget (UatF) technique, which provides a closedform expression for the achievable rate. The sum rate fairness trade-off optimization problem is subsequently solved through a sequential convex approximation (SCA) technique. Numerical results demonstrate that the proposed algorithms outperform conventional power control algorithms in terms of both the sum rate and minimum user rate. Furthermore, the TD3-based approach can increase the median of sum rate by 16%-46% and the median of minimum user rate by 11%-60% compared to the proposed SCA-based technique. Finally, we investigate the complexity and convergence of the proposed scheme. cc Index terms-Cell-free massive MIMO, deep reinforcement learning, fairness, power control, sequential convex approximation.

Joint User Association and Power Control for Cell-Free Massive MIMO

“…The proposed DNN model uses supervised learning to emulate Algorithm 1. It is worth mentioning that supervised learning is easy to train and deploy compared with unsupervised learning, while the latter is sometimes unstable, and the performance largely relies on finding the right parameters [32]. The main idea of the developed scheme is to learn the unknown mapping function between the LSF coefficients, PC, and UA.…”

“…Furthermore, unsupervised learning models can obtain solutions without prepared training datasets [25]- [27]. However, training these models is challenging since the performance largely depends on finding suitable parameters [32]. In a nutshell, to obtain the solutions for a joint UA and PC problem, iterative-based algorithms are unscalable for large-scale CFmMIMO systems, while DNN-based models rely on generating massive training examples and designing efficient training algorithms.…”

“…Update x(n) = x * 6: until convergence plemented to solve the problem (32). Starting from a random point x ∈ F, we solve problem (42) to obtain its optimal x * .…”

Uncoordinated Transmissions in Uplink IoT Cell-Free Massive MIMO Systems Based on NOMA

“…In (7), η k,am is the collision indicator for k on arm a m , which is zero in case of a power collision (i.e. 2 or more users choosing the same power level on a subband or when Γ k,am < Γ r ), and 1 otherwise.…”

“…Several previous works dealt with resource allocation in the context of uplink CF-mMIMO. In [7], deep reinforcement learning and sequential convex approximation (SCA) are used to solve the sum-rate fairness trade-off power optimization problem. The work in [8] proposes low-complexity solutions based on deep learning for solving the max-min, max-product, and max-sum-rate power control problems.…”