Tourists’ good brand sensory image for Traditional Chinese Medicine health tourism (TCMHT) can improve the brand building of tourism destinations. This paper takes some tourists who enter the scenic spots of integrated TCMHT as the research object to investigate the sensory perception of the brand of the TCMHT destination of this group, aiming to further analyze the matching degree between the existing brand building of TCMHT destination and Tourists’ Brand Sensory Impression (TBSI). This paper uses Matlab software to carry out a visual word cloud analysis on the TCMHT keywords that tourists pay more attention to. Based on the word cloud data, the binary Logistic regression model is constructed by using Python and SPSS software, and the influencing factors of TBSI’s selection of TCM are analyzed from six dimensions. The results show that the richness of natural landscape, the quantity and type of cultural resources, ecological environment, professional service ability, healthy diet and supporting facilities, and market position have a significant positive impact on the choice of TCMHT destination. Meanwhile, the marketing publicity, forms and features of TCMHT destinations have a significant positive impact on tourists’ impression intention. Based on the research conclusions and the characteristics of TBSI, this paper proposes the development positioning of tourist destinations, and provides theoretical guidance for the brand building of tourist destinations by using tourists’ unique, specific and direct sensory experience.