Hemicellulose is an important polysaccharide in ruminant nutrition, but it has not been studied as thoroughly as cellulose. Further research is needed to explore supplements that can improve its digestibility and ruminal buffering effects. Our previous research demonstrated the efficacy of oxalic acid (OA) as an essential nutrient in yeast culture (YC) for improving rumen fermentation performance. Consequently, we conducted in vitro rumen digestion experiments to examine the effects of YC and OA on rumen fermentation and bacterial composition. Two diets containing different levels of hemicellulose were formulated: diet 1 with 10.3% and diet 2 with 17% hemicellulose. Three levels of YC (0.00, 0.625, and 1.25 g/kg) and three doses of OA (0.0, 0.4, and 0.8 g/kg, DM) were added into each diet with a 3 × 3 factorial design. A comprehensive assessment was conducted on a total of 18 experimental treatments at fermentation periods of 0, 6, 12, 24, and 48 h. In the first experiment (diet 1), the supplementation of YC, OA, and their interaction significantly increased in vitro DM disappearance (IVDMD) and NDF disappearance (IVNDFD; p < 0.001). In the second experiment (diet 2), the supplementation of OA and the interaction between YC and OA (p < 0.001) increased IVDMD and IVCPD, but had no significant effects on IVNDFD. The interactions of YC and OA significantly increased ammonia nitrogen (p < 0.001). The production of acetic acid, propionic acid, and total volatile fatty acids (TVFA), and pH levels were significantly higher in treatments supplemented with YC and OA (p < 0.001). YC and OA in both diets significantly altered the rumen bacterial community leading to increased Shannon and Simpson diversity indices (p < 0.001). In both diets, OA supplementation significantly increased the relative abundance of the phylum Bacteroidetes and Prevotella genus. The result also showed a positive correlation between the Prevotella and Selenomonas genera with IVDMD, IVNDFD, propionic acid, and TVFA production, suggesting that these dominant bacteria enhanced nutrient disappearance in the rumen. In conclusion, adding YC and OA resulted in modifications to the bacterial community’s composition and diversity, and improved nutrient disappearance. These changes indicate improved rumen fermentation efficiency, which is promising for future in vivo studies.