Immunotherapy has revolutionized the treatment paradigm of many cancers, however, its effectiveness in prostate cancer patients is still under question. In the present systematic review and meta-analysis, we sought for assessing the efficacy and safety of Immune checkpoint inhibitors (ICIs) in patients with prostate cancer. PubMed, Scopus, Web of Science, and EMBASE databases were searched on Aguste 19, 2022. Thirty five studies met the eligibility criteria. The median overall survival (mOS) of all treatments was 14.1 months, with the longest and shortest mOS was seen among patients who received anti-CTLA-4 monotherapy and anti-PD-1/PD-L1+anti-CTLA-4 regimen at 24.9 and 9.2 months, respectively. Noteworthy, all types of adverse events had the lowest incidence in the anti-PD-1/PD-L1 monotherapy group. Considering the ICI monotherapy regimens, we found that fatigue, diarrhea, and infusion reaction had the highest incidence rates. Future studies evaluating the efficacy and safety of novel combination therapies with ICIs are warranted.