In this paper, we propose a novel learning-aided sphere decoding (SD) scheme for large multipleinput-multiple-output systems, namely, deep path prediction-based sphere decoding (DPP-SD). In this scheme, we employ a neural network (NN) to predict the minimum metrics of the "deep" paths in subtrees before commencing the tree search in SD. To reduce the complexity of the NN, we employ the input vector with a reduced dimension rather than using the original received signals and full channel matrix. The outputs of the NN, i.e., the predicted minimum path metrics, are exploited to determine the search order between the sub-trees, as well as to optimize the initial search radius, which may reduce the computational complexity of SD. For further complexity reduction, an early termination scheme based on the predicted minimum path metrics is also proposed. Our simulation results show that the proposed DPP-SD scheme provides a significant reduction in computational complexity compared with the conventional SD algorithm, despite achieving near-optimal performance.
Index TermsMIMO, sphere decoding, tree search, machine learning, deep learning, neural network.The authors are with the 2 interest [2]. To maximize the achievable data rates in large MIMO systems, the base station needs to receive as many symbols as possible simultaneously from multiple terminals, which leads to enhanced multiplexing gains. In this circumstance, a near-optimal receiver like SD plays an important role in approaching the channel capacity. However, the complexity of SD significantly increases with the number of antennas [3], which makes it difficult to apply to large MIMO systems.Recently, deep-learning (DL) techniques have been applied in various fields, exhibiting eminent performance. Motivated by the performance of DL technologies in other fields, there have been attempts to apply DL to MIMO detection [4]-[8]. In particular, the DL-based sphere decoding (DL-SD) algorithm is derived to choose the optimal hypersphere radius [4]. In addition, a deep network architecture, called DetNet, is proposed to estimate the solution of MIMO detection [5]. Furthermore, the sparsely connected neural network (ScNet) is developed to simplify the structure of DetNet for massive MIMO systems [6]. The application of a deep neural network to reduce the computational complexity of the conventional belief propagation detector is proposed in [7], and the orthogonal approximate message-passing network (OAMP-Net) architecture is proposed to improve the performance of the iterative detection algorithm with trainable variables [8].In this paper, a novel learning-aided SD algorithm is proposed. The main idea of the proposed algorithm is to predict the minimum path metric among "deep" paths of each sub-tree in a large tree structure by using a neural network (NN). In large MIMO systems, the required information to estimate the path metrics can have large dimension, which can significantly increase the complexity of the NN. To resolve this problem, the size of the input vector to...