Thermokarst lakes are forming from permafrost thaw and severely affected by accelerating climate change. Sediment and water in these lakes are distinct habitats but closely connected. However, our understanding of the differences and linkages between sediment and water in thermokarst lakes remain largely unknow, especially from the perspective of bacterial community patterns and underlying mechanisms. In this study, we examined bacterial communities in sediment and water in thermokarst lakes in the Yellow River Source area, Qinghai-Tibet Plateau. Bacterial taxa were divided to abundant and rare according to their relative abundance, and the Sorensen dissimilarity (βsor) was partitioned into turnover (βturn) and nestedness (βnest). The results showed that the whole bacterial communities as well as the abundant and rare subcommunities differed substantially between sediment and water, in terms of taxonomical composition, α-diversity, and β-diversity. Sediment had significantly lower α-diversity indexes but higher β-diversity than water. Abundant taxa had significantly higher relative abundances but lower α-diversity and β-diversity than rare taxa. Moreover, bacterial communities are predominantly governed by strong turnover processes (βturn/βsor ratio of 0.925). Abundant subcommunities were significantly lower in βturn/βsor ratio compared to rare subcommunities. Bacterial communities in sediment had a significantly higher βturn/βsor ratio than in water. The results suggest that the bacterial communities of thermokarst lakes, especially rare subcommunities or particularly in sediment, might be strongly structured by environmental filtering and geographical isolation, leading to compositional distinct. By revealing bacterial communities in sediment and water, this integral study increased our current knowledge of thermokarst lakes, enhancing our understanding of the community assembly rules and ecosystem structures and processes of these rapid changing and vulnerable ecosystems.