Glycosylation is one of the most important post-translational modifications of proteins and plays an essential role in spermatogenesis, maturation, extracellular quality control, capacitation, sperm−egg recognition, and final fertilization. Spermatozoa are synthesized in the testes inactively with a thick glycocalyx and passed through the epididymis for further modification by glycosylation, deglycosylation, and integration to reach maturation. Subsequently, sperm capacitation and further fertilization require redistribution of glycoconjugates and dramatic glycocalyx modification of the spermatozoa surface. Furthermore, glycoproteins and glycans in seminal plasma are functional in maintaining spermatozoa structure and stability. Therefore, aberrant glycosylation may cause alteration of semen function and even infertility. Currently, mass spectrometry-based technologies have allowed large-scale profiling of glycans and glycoproteins in human semen. Quantitative analysis of semen glycosylation has also indicated many involved glycoproteome issues in male infertility and the potential biomarkers for diagnosis of male infertility in clinical. This review summarizes the role of glycosylation during spermatozoa development, the large-scale profiling of glycome and glycoproteome in human semen, as well as the association of aberrant glycosylation with infertility.