Many studies have suggested that SARS-CoV-2, directly or indirectly, can affect the male reproductive system, although the underlined mechanisms have not been completely elucidated yet. The purpose of this review is to provide a summary of the current data concerning the impact of SARS-CoV-2 infection on the male urogenital tract, with a particular emphasis on the testes and male fertility. The main data regarding the morphological alterations in the testes emerged from autoptic studies that revealed interstitial congestion, micro thrombosis, reduction of Sertoli, Leydig, and germinal cells, infiltrated immune cells, and atrophic seminiferous tubules consistent with orchitis. Furthermore, men with severe infection exhibit sperm parameter alterations, together with abnormalities of the hypothalamic–pituitary–testis axis, strongly suggesting that SARS-CoV-2 could increase the risk of male infertility. However, despite the inadequate number of longitudinal studies, spermatogenesis and sex hormone imbalance seem to improve after infection resolution. The yet unresolved question is whether the virus acts in a direct or/and indirect manner, as discordant data related to its presence in the testis and semen have been reported. Regardless of the direct effect, it has been postulated that the cytokine storm and the related local and systemic inflammation could strongly contribute to the onset of testis dysfunction, leading to male infertility. Therefore, multicentric and longitudinal studies involving a large number of patients are needed to understand the real impact of SARS-CoV-2 infection on male reproduction.