In China, the opening of high-speed rails (HSR) brings significant changes to the source-destination spatial distance, the accessibility of destinations, and the spatial structure of tourist flows in each region, exerting varied HSR effects on different types of cities. Against this backdrop, it is meaningful to deeply explore tourists’ preference for city destinations in the light of HSR effects. The exploration could contribute greatly to the planning, marketing, management, and sustainable development of urban tourism. This paper takes Xiangtan and Yueyang as typical cases of the diffusion effect and the corridor effect of HSR. Firstly, the factors affecting destination choice were identified, and the attribute levels were configured, forming multiple virtual alternatives. Next, questionnaire surveys were carried out to collect tourists’ selections between each pair of alternatives. Further, a discrete choice model was constructed to assign a weight to each factor, reflecting its importance to tourists’ decision-making regarding their destination selection and to disclose the law of tourists’ preferences for destinations. The results showed that (1) Under the HSR diffusion effect, the top three factors affecting tourists’ preference for destinations in Xiangtan are convenience, connection time, and popularity; under HSR corridor effect, the top three factors affecting the tourists’ preference for destinations in Yueyang are reputation, convenience, and leisure and reception facilities (LRFs). (2) The destination preference is closely associated with personal features like gender, income, occupation, and fellow travelers. Tourists with different personal features give different attention to the various influencing factors. The research findings provide a reference for the sustainable development of urban tourism.