This study aimed to explore the effects of different levels of barley starch instead of corn starch on the rumen fermentation and microflora when feeding a corn-based diet to Hu sheep. Thirty-two male Hu sheep equipped with permanent rumen fistulas were selected and fed in individual metabolic cages. All sheep were randomly divided into four groups (eight sheep in each group) and fed with four diets containing a similar starch content, but from different starch sources, including 100% of starch derived from corn (CS), 33% of starch derived from barley + 67% of starch derived from corn (33 BS), 67% of starch derived from barley + 33% of starch derived from corn (67 BS) and 100% of starch derived from barley (100 BS). The experimental period included a 14 d adaptation period and a 2 d continuous data collection period. The results showed that the molar proportions of acetate, isobutyrate, butyrate and isovalerate and the ratio of acetate to propionate in the 67 BS and 100 BS groups decreased compared with the CS and 33 BS groups (p < 0.001), while the molar proportions of propionate and valerate increased (p < 0.001). The combination of 33% barley starch and 67% corn starch in the diet improved the production of TVFAs (p = 0.007). The OTUs and Shannon indexes of the CS and 33 BS groups were higher than the 67 BS and 100 BS groups (p < 0.001), and the Chao1 and Ace indexes were higher than the 100 BS group (p < 0.05). In addition, the 33 BS group had increased the relative abundances of Bacteroidetes, Prevotella and Ruminococcus and the abundances of Fibrobacter succinogenes, Ruminococcus flavefaciens, Streptococcus bovis, Selenomonas ruminantium and Prevotella brevis relative to the CS group (p < 0.05). These results indicate that the substitution of 33% of the CS with BS did not change the rumen fermentation pattern relative to the CS group, and increased the richness and diversity of the rumen microbes in Hu sheep compared with other two starch substitute groups.