Age is an important factor in shaping the gut microbiome. However, the age effect on the rumen microbial community for dairy buffaloes remains less explored. Using metagenomics, we examined the microbial composition and functions of rumen microbiota in dairy Murrah buffaloes of different ages: Y (1 year old), M (3–5 years old), E (6–8 years old), and O (>9 years old). We found that Bacteroidetes and Firmicutes were the predominant phyla, with Prevotella accounting for the highest abundance at the genus level. The proportion of Bacteroides and Methanobrevibacter significantly increased with age, while the abundance of genus Lactobacillus significantly decreased with age (LDA > 3, p < 0.05). Most differed COG and KEGG pathways were enriched in Y with carbohydrate metabolism, while older buffaloes enriched more functions of protein metabolism and the processing of replication and repair (LDA > 2, p < 0.05). Additionally, the functional contribution analysis revealed that the genera Prevotella and Lactobacillus of Y with more functions of CAZymes encoded genes of glycoside hydrolases and carbohydrate esterases for their roles of capable of metabolizing starch and sucrose-associated oligosaccharide enzyme, hemicellulase, and cellulase activities than the other three groups (LDA > 2, p < 0.05), thus affecting the 1-year-old dairy buffalo rumen carbohydrate metabolism. This study provides comprehensive dairy buffalo rumen metagenome data and assists in manipulating the rumen microbiome for improved dairy buffalo production.