The prevalence of depression and anxiety in the Chinese male population with infertility is still uncertain. The prevalence of depression, anxiety, and a combination of both psychological symptoms was 20.8%, 7.8%, and 15.4%, respectively in 771 infertile Chinese men in the current study by the Mental Health Inventory–5 and the State–Trait Anxiety Inventory–Short Form questionnaires. Differences in demographics (age, education, and income) had no noticeable impact on the development of psychological symptoms. Clinical factors such as concomitant disorders (varicocele, epididymal cyst, and erectile dysfunction) were identified as risk factors associated with depressive symptoms (OR = 1.47; 95% CI [1.14, 1.90]; p < .001) and both depressive and anxiety symptoms (OR = 1.56; 95% CI [1.17, 2.08]; p < .001). An infertility duration over 2 years was associated with a high risk of anxiety symptoms (OR = 3.94; 95% CI [1.20, 12.93], p < .02). Other clinical conditions such as type of treatment and quality of sperm were not significant risk factors for psychological symptoms. This study provides evidence that Chinese men of reproductive age who suffer from infertility are vulnerable to psychological distress.