World Heritage Sites (WHS) possess outstanding universal value (OUV) centered on science and aesthetics, and the large scale of tourism has a certain influence on sustainable development, which will have some degree influence on the quality of tourist service and experience. Taking the World Natural Heritage Site Mount Sanqingshan National Park in China as a case, we collected 535 samples of tourists and used structural equation modeling as a methodology to construct a theoretical framework from the perspective of tourists’ perception, including tourism crowding (functional crowding, personal crowding, and social crowding), service quality (interpretation, goods, commuting, accommodation, and catering), tourists’ satisfaction, and OUV attractiveness as intermediary variables. The results were: (1) tourist crowding has a substantial negative influence on satisfaction, and the negative influence on OUV attractiveness is not tenable; (2) service quality has a substantial positive impact on OUV attractiveness and satisfaction; (3) the tourists’ perception of OUV attractiveness has a mediating influence on service quality and satisfaction, but there is no mediating impact on tourism crowding and satisfaction; (4) this paper puts forward the SCA-S (service, crowding, attractiveness, and satisfaction) framework of WHS, and explores impact factors and mechanisms of visitor satisfaction of WHS from different aspects. The relevant research conclusions have some theoretical value and practical significance for the interpretation and display of heritage value, improved service quality, and tourism experience, and they are conducive to protecting WHS.