Deterioration development is a recognized worldwide threat to rock carvings, especially in humid regions of southern China. Cultural heritage managers thus require precise identification of different deterioration patterns and conduct comprehensive assessments. However, the quantitative analysis of deterioration patterns is limited due to the severe impact of temperature and humidity on rock carvings. Additionally, the current research on the different deterioration patterns is independent, and the corresponding systematic framework is vague. Based on this, the hyperspectral response is constructed to evaluate the various deterioration patterns using spectral index and intelligent model. Firstly, the remarkable correlation between the feldspar content and the deterioration patterns of rock carvings with the influence of environmental factors is investigated by mineralogical analysis. Secondly, combined with microscopic and mineralogical characteristics, the extracted deterioration characteristics are qualitatively screened. Then, a novel spectral index characterizing the correlation between image grayscale and spectral reflectance is proposed by introducing dynamic correction, and the optimal wavelength combination is applied to identify the distribution of deterioration patterns. Consequently, the quantitative screening of deterioration patterns can be realized. After that, the WOA-XGBoost model exhibits better performance in the classification of deterioration patterns. Finally, the influence of different deterioration patterns on rock carvings is quantified by integrating the deterioration index reflected by chemical composition and the proportion of deterioration pattern distribution identified by the spectral response. In the regional deterioration assessment of Dazu Rock Carvings, biological colonization and surface morphological changes have the highest proportion and degree of deterioration, which is worthy of attention in the protection of rock carvings in this region.