Infrared small target detection is a crucial and challenging topic for various applications. In recent years, the spectrum scale space (SSS) algorithm has shown considerable potential in the field of target detection. However, the SSS algorithm is prone to high false alarm rates in infrared small target detection scenarios with complex background. This paper proposes an improved SSS (ISSS) algorithm via precise feature matching and scale selection strategy for efficient infrared small target detection, which includes background suppression, feature matching and optimal scale selection three stages. In the background suppression stage, a matrix decomposition method named inexact augmented Lagrange multiplier (IALM) algorithm is used to extract the sparse image matrix from the original image as the target foreground image. In the feature matching stage, the 16 elaborate Gaussian kernel functions convolve with the the amplitude spectrum of target foreground image to generate 16 scale saliency maps that precisely match the feature of small targets. In the optimal scale selection stage, a few proper candidate scale maps are screened out according to the difference between the pixel values of the target area and the background clutters, in which the target area was more highlighted, and the scale map corresponding to the maximum value of local information entropy of the candidate saliency map is the final detection result map. We mainly made three contributions: First, IALM algorithm is utilized as a preprocessing step, and we have verified it is indispensable in eliminating most backgrounds with self-correlation property. Second, an elaborate scale division strategy is proposed to obtain multi-scale saliency maps that match the feature of infrared small targets precisely. Third, the gray value difference and the maximum value of local information entropy are defined and used as the judgment criteria for optimal scale selection. Extensive experimental results demonstrate that the proposed method outperforms state-of-the-art techniques, especially on infrared images with thick clouds and high-brightness buildings. INDEX TERMS Infrared small target detection, improved spectrum scale space, matrix decomposition.