The structure and types of rumen microbes are closely related to host health. This study aimed to evaluate the effect of Broussonetia papyrifera silage (BPS) gradually replacing the whole crop maize silage (WCMS) on total tract digestibility, rumen fermentation parameters, serum biochemical indicators, and rumen microbes of Holstein heifers. The diet treatment consisted of four proportions of BPS (0%, 25%, 50% and 75%) as substitute for WCMS (designated as T0, T25, T50 and T75, respectively). Twenty heifers (body weight = 245 ± 24 kg) were randomly divided into four groups of five heifers, and randomly received one diet. The feeding adaption period was 7 days, and the experiment period was 30 days. Our findings suggested that the digestibility of neutral detergent fiber and crude protein increased linearly with the increased in BPS (P < 0.05). The concentrations of total protein and albumin increased quadratically with the increased in BPS (P < 0.05). The 16s high-throughput sequencing showed that feeding BPS did not change the diversity and structure of the rumen microbes of heifers. However, the relative abundances of Tenericutes and SR1–Absconditabacteria increased linearly with the increased in BPS (P < 0.05). The Weighted Correlation Network Analysis results suggested that ALT concentration was positively correlated with the abundance of Prevotella-1 (r = 0.73; P = 0.007). In general, Holstein heifers fed with BPS did not change the diversity and uniformity of rumen microbes, and enhanced the body protein metabolism.