Despite the crucial role of the gut microbiota in different physiological processes occurring in the animal body, reports regarding the gut microbiota of animals residing in different environmental conditions like high altitude and different climate settings are limited. The Qinghai–Tibetan Plateau is renowned for its extreme climatic conditions that provide an ideal environment for exploring the effects of high altitude and temperature on the microbiota of animals. Yaks have unique oxygen delivery systems and genes related to hypoxic response. Damxung, Nyêmo, and Linzhou counties in Tibet have variable altitudes and temperatures that offer distinct settings for studying yak adaptation to elevated terrains. The results of our study suggest that amplicon sequencing of V3-V4 and internal transcribed spacer 2 (ITS2) regions yielded 13,683 bacterial and 1912 fungal amplicon sequence variants (ASVs). Alpha and beta diversity indicated distinct microbial structures. Dominant bacterial phyla were Firmicutes, Bacteroidota, and Actinobacteriota. Genera UCG-005, Christensenellaceae_R-7_group, and Rikenellaceae_RC9_gut_group were dominant in confined yaks living in Damxung county (DXS) and yaks living in Linzhou county (LZS), whereas UCG-005 prevailed in confined yaks living in Nyêmo county (NMS). The linear discriminant analysis effect size (LEfSe) analysis highlighted genus-level differences. Meta-stat analysis revealed significant shifts in bacterial and fungal community composition in yaks at different high altitudes and temperatures. Bacterial taxonomic analysis revealed that two phyla and 32 genera differed significantly (p < 0.05). Fungal taxonomic analysis revealed that three phyla and four genera differed significantly (p < 0.05). Functional predictions indicated altered metabolic functions, especially in the digestive system of yaks living in NMS. This study reveals significant shifts in yak gut microbiota in response to varying environmental factors, such as altitude and temperature, shedding light on previously unexplored aspects of yak physiology in extreme environments.