While previous research shows that bilinguals' ability to mix and switch between two separate languages is influenced by both cross-language similarity and language use contexts, little attention has been paid to bidialectal populations. Given the linguistic and sociolinguistic differences between bilingualism and bidialectalism, it is thus unclear to what extent mechanisms underlying bidialectal variety switching resemble those underlying bilinguals. To investigate the effects of cross-variety phonological distance and variety use contexts on variety switching, we tested two groups of Chinese speakers in a bidialectal auditory word recognition task. Both groups speak a regional dialect (Chengdu Mandarin or Cantonese) as their native language and are also highly proficient in Standard Mandarin. Participants' language background and linguistic experience are collected by a language background questionnaire. Mixing and switching between two varieties are costly for bidialectal speakers in comprehension. Mixing costs can be attributed to variety switches instead of the mere presence of a mixed-variety context. While variety switch and mixing costs are not influenced by cross-variety phonological distance, they are modulated by variety dominance and habitual variety use contexts. This study highlights the similarities between bidialectals and bilinguals in language processing, as well as the significance of recognizing between- and within-group differences in conducting psycholinguistics research with multilingual populations.