Salt crystallization represents one of the primary forms of weathering encountered in rock cultural heritage sites, with sulfate weathering having particularly notable destructive effects. This study focuses on sandstone and limestone, using them as test materials to conduct simulation experiments on sulfate weathering under specific environmental conditions. The experimental process involved documenting the surface morphology of the rock samples and analyzing changes in indicators such as wave velocity, hardness, composition, and pore size distribution. The degree of damage to the two types of rock was evaluated using the entropy weight–TOPSIS method, and the sensitivity of different weathering indicators in assessing the weathering of the two rocks was also discussed. The results revealed that sandstone exhibited obvious surface damage under sulfate erosion, with dissolution holes and pits surrounding the rock samples, while limestone primarily suffered damage at its edges. There were notable differences in the rate of attenuation observed in the macro and micro indicators between the two rock types. The wave velocity of both types of rocks exhibits linear attenuation while the intensity undergoes exponential change. It is worth noting that sandstone hardness demonstrates a pattern of “fast–rapid–slow–stable” decline characteristics, whereas limestone follows an exponential trend with an initial fast decline followed by a slower decline. Additionally, sandstone exhibited significantly greater damage and weathering thickness compared to limestone, owing to the involvement of complex and diverse physical and chemical reactions. The pore damage factor and macro-level indicators of the rock samples could be fitted using exponential and linear functions, respectively, although the fitting curves differed distinctly. The sensitivity indicators reflecting the weathering state of sandstone and limestone under sulfate erosion varied, with mass loss applicable to sandstone and porosity to limestone. Overall, with our research findings, we aim to provide a theoretical foundation for the anti-salination and precision protection of rock cultural heritage sites.