Background: Sex is an important factor influencing the immune system, and the distribution of tumors, including their types and subtypes, is characterized by sexual dichotomy. The aim of this study was to investigate whether there is an association between sex and the treatment effect of immune checkpoint inhibitors (ICI). Methods: Four bibliographic databases were searched. Studies of randomized controlled trials (RCTs) assessing the efficacy of ICI were identified and used, and the primary endpoint was the difference in efficacy of ICI between males and females, presented as overall survival (OS), progression-free survival (PFS) and recurrence-free survival (RFS). The study calculated the pooled HRs and 95% CIs for OS, PFS and RFS for males and females using a random effects model or a fixed effects model, and thereby assessed the effect of sex on the efficacy of ICI treatment. This study is registered with PROSPERO (CRD42022370939). Results: A total of 103 articles, including a total of 63,755 patients with cancer, were retrieved from the bibliographic database, of which approximately 70% were males. In studies with OS as the outcome, the combined hazard ratio (HR) was 0.77 (95% CI 0.74–0.79) for male patients treated with ICI and 0.81 (95% CI 0.78–0.85) for female patients compared to controls, respectively. The difference in efficacy between males and females was significant. Conclusions: ICI therapy, under suitable conditions for its use, has a positive impact on survival in various types of tumors, and male patients benefit more than females. It may be necessary to develop different tumor immunotherapy strategies for patients of different sexes.