The magnetic anomalies generated by the ferromagnetic targets are usually buried within uncontrollable interference sources, such as the power frequency and random noises. In particular, the variability of the geomagnetic field and the low signal-to-noise ratio (SNR) of the magnetic anomalies cannot be avoided. In this paper, to improve the performance of magnetic anomaly detection (MAD) with a low SNR, we propose a novel structured low-rank (SLR) decomposition-based MAD method. In addition, a new framework based on the SLR and singular value decomposition (SVD) is constructed, dubbed SLR-SVD, and the corresponding working principle and implemented strategy are elaborated. Through comparing the SLR-SVD with two state-of-the-art methods, including principal component analysis and SVD, the results demonstrate that the proposed SLR-SVD can not only suppress the noise sufficiently, i.e., improving 55.26% approximately of the SNR, but also retain more boundary information of magnetic anomalies, i.e., decreasing approximately 68.05% of the mean squared error and improving approximately 28.47% of the structural similarity index.