The rumen is an important digestive organ of ruminants. From the fetal to adult stage, the morphology, structure and function of the rumen change significantly. However, the knowledge of the intrinsic genetic regulation of these changes is still limited. We previously reported a genome-wide expression profile of miRNAs in pre-natal goat rumens. In this study, we combined and analyzed the transcriptomes of rumen miRNAs during pre-natal (E60 and E135) and post-natal (D30 and D150) stages. A total of 66 differentially expressed miRNAs (DEMs) were identified in the rumen tissues from D30 and D150 goats. Of these, 17 DEMs were consistently highly expressed in the rumens at the pre-weaning stages (E60, E135 and D30), while down-regulated at D150. Noteworthy, annotation analysis revealed that the target genes regulated by the DEMs were mainly enriched in MAPK signaling pathway, Jak-STAT signaling pathway and Ras signaling pathway. Interestingly, the expression of miR-148a-3p was significantly high in the embryonic stage and down-regulated at D150. The potential binding sites of miR-148a-3p in the 3′-UTR of QKI were predicted by the TargetScan and verified by the dual luciferase report assay. The co-localization of miR-148a-3p and QKI through in situ hybridization was observed in the rumen tissues but not in the intestinal tracts. Moreover, the expression of miR-148a-3p in the epithelium was significantly higher than that in the other layers of the rumen, suggesting that miR-148a-3p is involved in the development of the rumen epithelial cells by targeting QKI. Subsequently, miR-148a-3p inhibitor was found to induce the proliferation of GES-1 cells. Taken together, our study identified DEMs involved in the development of the rumen and provides insights into the regulation mechanism of rumen development in goats.