Multi-source remote sensing data fusion could compensate for the lack of spatial or spectral resolution of single-source remote sensing images. In this paper, we took the rocky area as the research object and carried out a machine learning lithology classification study based on combining spatial features and spectral features, and proposed an automatic identification method of surface lithology based on limit learning and watershed segmentation. By comparing and analyzing typical machine learning algorithms, the combination of spectral features and spatial features was incorporated into the algorithm to form a set of high-precision lithology automatic identification methods. The experimental results showed that the method in this paper performed better than the traditional SVM classification method based on spectral features in terms of classification accuracy and Kappa coefficient, especially after the extreme learning fusion watershed segmentation algorithm.