Introduction: Testicular microlithiasis (TML) was shown to be associated with an increased risk of infertility. However, the association of TML with spermatogenesis in patients with unexplained infertility is still unknown. In this study, we therefore investigated the effect of TML on hormones and sperm parameters in a large cohort of infertile men without major factors for impaired fertility and azoospermic men serving for comparison. Methods: Over a period of 10 years, we retrospectively analyzed 2,914 patients who attended our centre with the diagnosis of unexplained infertility and sperm count >1 million/ejaculate, as well as 281 patients with unexplained azoospermia. From the 2,914 patients, we identified 218 patients with TML as revealed by ultrasound imaging. Further, 26 out of 281 azoospermic patients showed TML. Subsequently, we performed a thorough analysis of reproductive parameters and their association with TML. Results: The overall incidence of TML in patients with unexplained infertility and in unexplained azoospermic men was 7.5 and 9.3%, respectively. Patients with unexplained infertility and TML showed significantly smaller testicular volume, elevated FSH level, and lower sperm count and motility. Impaired spermatogenesis was not associated with the amount of microlithiasis, considered after classification into subgroups (<5 vs. ≥5 microliths/testis), and instead was associated with presence or absence of TML. TML in unexplained infertile azoospermic patients was not significantly associated neither with andrological reproductive parameters nor with sperm retrieval rate in microsurgical testicular sperm extraction. Discussion/Conclusion: TML itself, and not the number of microliths, is associated with impaired spermatogenesis in patients with unexplained infertility. The parameter TML alone is not sufficient to predict spermatogenic impairment in azoospermic patients. This study highlights the importance of ultrasound imaging in the clinical evaluation of infertile men, taking into account that TML is a negative co-factor for male fertility.