Obtaining accurate angle parameters using direction-of-arrival (DOA) estimation algorithms is crucial for acquiring channel state information (CSI) in massive multiple-input multiple-output (MIMO) systems. However, the performance of the existing algorithms deteriorates severely due to mutual coupling between antenna elements in practical engineering. Therefore, for solving the array mutual coupling, the array output signal vector is modeled by mutual coupling coefficients and the DOA estimation problem is transformed into block sparse signal reconstruction and parameter optimization in this paper. Then, a novel sparse Bayesian learning (SBL)-based algorithm is proposed, in which the expectation-maximum (EM) algorithm is used to estimate the unknown parameters iteratively, and the convergence speed of the algorithm is enhanced by utilizing the approximate approximation. Moreover, considering the off-grid error caused by discretization processes, the grid refinement is carried out using the polynomial roots to realize the dynamic update of the grid points, so as to improve the DOA estimation accuracy. Simulation results show that compared with the existing algorithms, the proposed algorithm is more robust to mutual coupling and off-grid error and can obtain better estimation performance.

No abstract