The murine epididymis has 10 distinct segments that provide the opportunity to identify compartmentalized cell physiological mechanisms underlying sperm maturation. However, despite the essential role of the epididymis in reproduction, remarkably little is known about segment-specific functions of this organ. Here, we investigate the dramatic segmental localization of the ganglioside GM1, a glycosphingolipid already known to play key roles in sperm capacitation and acrosome exocytosis. Frozen tissue sections of epididymides from adult mice were treated with the binding subunit of cholera toxin conjugated to AlexaFluor 488 to label GM1. We report that GM1-enriched vesicles were found exclusively in principal and clear cells of segment 2. These vesicles were also restricted to the lumen of segment 2 and did not appear to flow with the sperm into segment 3, within the limits of detection by confocal microscopy. Interestingly, this segment-specific presence was altered in several azoospermic mouse models and in wild-type mice after efferent duct ligation. These findings indicate that a lumicrine factor, itself dependent on spermatogenesis, controls this segmental differentiation. The RNA sequencing results confirmed global de-differentiation of the proximal epididymal segments in response to efferent duct ligation. Additionally, GM1 localization on the surface of the sperm head increased as sperm transit through segment 2 and have contact with the GM1-enriched vesicles. This is the first report of segment-specific vesicles and their role in enriching sperm with GM1, a glycosphingolipid known to be critical for sperm function, providing key insights into the segment-specific physiology and function of the epididymis.