The aim of this study is to determine the effects of a high-concentrate diet on growth performance, serum biochemical indexes, and rumen microbiota in house-fed yaks. Sixteen male yaks (body weight, 151.73 ± 14.11 kg; 18 months) were randomly allocated into two dietary treatments: a group with a low level of concentrate (n = 8, LC, concentrate–forage = 40:60) and a group with a high level of concentrate (n = 8, HC, concentrate–forage = 60:40). We found that compared with the LC group, the average daily feed intake (ADFI), the average daily gain (ADG), and the serum albumin (ALB) concentration in the HC group were significantly increased (p < 0.05). The rumen bacterial compositions also differed significantly between the groups, as indicated by principal coordinate analysis (p < 0.05). Firmicutes and Bacteroidota were the main dominant phyla of rumen bacteria in yaks. Compared with the LC group, the relative abundance of Firmicutes in the HC group was significantly increased (p < 0.05) and the relative abundance of Bacteroidota was significantly decreased (p < 0.05). At the genus level, Rikenellaceae_RC9_gut_group, Succiniclasticum, Prevotella, Christensenellaceae_R-7_group, and NK4A214_group had the highest relative abundance. The relative abundance of Christensenellaceae_R-7_group in the HC group was significantly higher than that in the LC group (p < 0.05). The PICRUSt 2 results showed a significant enrichment in glycosaminoglycan degradation, apoptosis, and ECM–receptor interaction in HC relative to LC (p < 0.05). In conclusion, high-concentrate diets can enhance growth performance and alter the compositions and functions of ruminal bacterial communities in yaks.