The Tianzhu white yaks (Bos grunniens) live on the Qinghai–Tibet Plateau. During winter, a lack of resources and low nutritional levels seriously affect their growth performance. In this study, we aimed to explore the effect of supplementation straw pellets on the growth performance and ruminal microbiota of yaks. Overall, at 6 (6M, n = 24), 18 (18M, n = 26), 30 (30M, n = 20), 42 (42M, n = 24), and 54 (54M, n = 22) month old Tianzhu white yaks were selected (total n = 116) and divided into the mixed straw + grazing (MSG), corn straw + grazing (CSG), and the grazing control (G) groups according to age and gender. Their growth performance was measured as per different dietary treatments. The rumen microbial community structure and levels of VFAs were analyzed from the 6M, 30M, and 54M male yaks from each group. The supplementary diets led to an increase in the ADG, which was the highest in the MSG group. The MSG group exhibited the highest level of acetate and total VFAs (TVFAs) among the three groups (p < 0.05). In addition, the 16S rRNA sequencing results proved that the microbial composition was dominated by the members of Firmicutes and Bacteroidetes. Christensenellaceae R-7 group was significantly abundant in the CSG and MSG groups compared to the G group (p < 0.05). Principal coordinate analysis (PCoA) revealed that the bacterial community structure of rumen in the MSG and CSG groups was considerably different from that in the G group; 6M samples exhibited different rumen microbial diversity compared with the other samples. Correlation analysis revealed that Christensenellaceae_R-7_group was positively correlated with the levels of acetate, TVFAs, and ADG. These results demonstrated that mixed straw pellets improved the growth performance of yaks, increased the abundance of Christensenellaceae R-7_group involved in cellulose degradation in the rumen, and produced large amounts of VFAs, which were absorbed by yaks, thus increasing their ADG. This study provides new insights into the effects of straw pellet supplementation on the changes in the rumen microbiota and growth performance of yaks.