In this study, we conducted comparative analyses to characterize the rumen microbiota and volatile fatty acid (VFA) profiles of weaned Nanjiang Yellow goat kids under shrub-grassland grazing (GR), shrub-grassland grazing and supplementary feeding (SF), and indoor feeding (IF) systems. We observed significant differences (p < 0.05) in the concentrations of total VFA and the proportions of acetate and butyrate in the rumen fluid among the three groups, whereas the proportions of propionate and the acetate/propionate ratio did not differ substantially. Alpha diversity of the rumen bacterial and archaeal populations in the GR and SF kids was significantly higher (p < 0.05) than that in the IF goat kids, and significant differences (p < 0.05) in similarity were observed in the comparisons of GR vs. IF and SF vs. IF. The most predominant bacterial phyla were Bacteroidetes and Firmicutes across the three groups, and the archaeal community was mainly composed of Euryarchaeota. At the genus and species levels, the cellulose-degrading bacteria, including Lachnospiraceae, Ruminococcaceae and Butyrivibrio fibrisolvens, were abundant in the GR and SF groups. Furthermore, 27 bacterial and 11 unique archaeal taxa, such as Lachnospiraceae, Butyrivibrio fibrisolvens, and Methanobrevibacter ruminantium, were identified as biomarkers, and showed significantly different (p < 0.05) abundances among the three groups. Significant Spearman correlations (p < 0.05), between the abundances of several microbial biomarkers and the concentrations of VFAs, were further observed. In summary, our results demonstrated that the adaptation to grazing required more rumen bacterial populations due to complex forage types in shrub-grassland, although the rumen fermentation pattern did not change substantially among the three feeding systems. Some microbial taxa could be used as biomarkers for different feeding systems, particularly cellulose-degrading bacteria associated with grazing.