Existing remote sensing images of ground objects are difficult to annotate, and building a hyperspectral dataset requires huge resources. To tackle these problems, this paper proposes a new method with low requirements for the scale of the dataset that involves correcting the inter-class differences of hyperspectral images and eliminating the redundant information of spatial–spectral features. Firstly, the algorithm introduces the spatial information of hyperspectral images into the classification task through the superpixel definition based on the entropy rate, which not only reduces the spatial redundancy information of hyperspectral images but also uses the similarity of spatial neighbors to increase the differences between classes and alleviate the differences within classes. Secondly, under the theoretical guidance of similar spectral fluctuation trends of similar objects, a feature extraction method based on fast Fourier transform is proposed to alleviate spectral feature redundancy and further eliminate the inter-class differences of the algorithm. Finally, to verify the improvement effect of the proposed idea on the traditional classification method, the idea was applied to the traditional SVM algorithm, and experiments were carried out based on the PaviaU and Indian Pines hyperspectral datasets. The simulation results show that the proposed (ERS–FFT–SVM) algorithm shows a significant improvement in classification accuracy when compared with the traditional classification algorithm and is able to perform the small sample classification of hyperspectral images.