Systemic inflammatory status has been reported to impact survival of prostate cancer (PCa) patients; however, evidence is lacking on whether the inflammatory potential of diet can influence prognosis of PCa patients. To investigate the association between a dietary inflammatory index (DII) and PCa survival, we conducted a retrospective cohort study including 726 men with PCa originally enrolled, between 1995 and 2002, in an Italian case–control study. Information on diet and Gleason score was collected at PCa diagnosis. DII was derived from a food frequency questionnaire using a validated algorithm. Adjusted hazard ratios (HRs) of death with 95% confidence intervals (CIs) were estimated using a Fine-Gray model. DII scores were not significantly associated with all-cause mortality of PCa patients (HR highest vs. lowest DII tertile = 1.25; 95% CI: 0.86–1.83). However, considerable heterogeneity emerged according to Gleason score (p < 0.01): no associations emerged among men with Gleason score 2–6 PCa; whereas, among patients with Gleason score 7–10 PCa, DII was directly associated with both all-cause and PCa-specific mortality (HR highest vs. lowest DII tertile: 2.78; 95% CI: 1.41–5.48; and 4.01; 95% CI: 1.25–12.86; respectively). Among patients with Gleason score 7–10 PCa, ten-year all-cause survival probabilities were 58% (95% CI: 47–67%) for highest and 78% (95% CI: 67–86%) for lowest DII tertile. Study findings support the hypothesis that diet, through its inflammatory potential, may influence the prognosis of patients with more aggressive PCa. Dietary interventions aimed at decreasing inflammation may be considered to improve survival of men with PCa.