Yanshan Cashmere bucks are seasonal breeding animals and an important national genetic resource. This study aimed to investigate the involvement of prolactin (PRL) in the epididymal function of bucks. Twenty eleven-month-old Cashmere bucks were randomly divided into a control (CON) group and a bromocriptine (BCR, a prolactin inhibitor, 0.06 mg/kg body weight (BW)) treatment group. The experiment was conducted from September to October 2020 in Qinhuangdao City, China, and lasted for 30 days. Blood was collected on the last day before the BCR treatment (day 0) and on the 15th and 30th days after the BCR treatment (days 15 and 30). On the 30th day, all bucks were transported to the local slaughterhouse, where epididymal samples were collected immediately after slaughter. The left epididymis was preserved in 4% paraformaldehyde for histological observation, and the right epididymis was immediately preserved in liquid nitrogen for RNA sequencing (RNA-seq). The results show that the PRL inhibitor reduced the serum PRL and estradiol (E2) concentrations (p < 0.05) and tended to decrease luteinizing hormone (LH) concentrations (p = 0.052) by the 30th day, but no differences (p > 0.05) occurred by either day 0 or 15. There were no differences (p > 0.05) observed in the follicle-stimulating hormone (FSH), testosterone (T), and dihydrotestosterone (DHT) concentrations between the two groups. The PRL receptor (PRLR) protein was mainly located in the cytoplasm and intercellular substance of the epididymal epithelial cells. The PRL inhibitor decreased (p < 0.05) the expression of the PRLR protein in the epididymis. In the BCR group, the height of the epididymal epithelium in the caput and cauda increased, as did the diameter of the epididymal duct in the caput (p < 0.05). However, the diameter of the cauda epididymal duct decreased (p < 0.05). Thereafter, a total of 358 differentially expressed genes (DEGs) were identified in the epididymal tissues, among which 191 were upregulated and 167 were downregulated. Gene Ontology and Kyoto Encyclopedia of Genes and Genomes analyses revealed that ESR2, MAPK10, JUN, ACTL7A, and CALML4 were mainly enriched in the estrogen signaling pathway, steroid binding, calcium ion binding, the GnRH signaling pathway, the cAMP signaling pathway, and the chemical carcinogenesis–reactive oxygen species pathway, which are related to epididymal function. In conclusion, the inhibition of PRL may affect the structure of the epididymis by reducing the expression of the PRLR protein and the secretion of E2. ESR2, MAPK10, JUN, ACTL7A, and CALML4 could be the key genes of PRL in its regulation of epididymal reproductive function.