Existing evidence suggests a link between the inflammatory potential of diet and risk of cancer. This study aimed to test the linear and potential nonlinear dose-response associations of the Dietary Inflammatory Index (DII), as being representative of inflammatory features of the diet, and site-specific cancer risk. A systematic search was conducted with the use of PubMed and Scopus from 2014 to November 2017. Prospective cohort or case-control studies reporting the risk estimates of any cancer type for ≥3 categories of the DII were selected. Studies that reported the association between continuous DII score and cancer risk were also included. Pooled RRs were calculated by using a random-effects model. Eleven prospective cohort studies (total n = 1,187,474) with 28,614 incident cases and 29 case-control studies with 19,718 cases and 33,229 controls were identified. The pooled RRs for a 1-unit increment in the DII were as follows: colorectal cancer, 1.06 (95% CI: 1.04, 1.08; I2 = 72.5%; n = 9); breast cancer, 1.03 (95% CI: 1.00, 1.07; I2 = 84.0%; n = 7); prostate cancer, 1.06 (95% CI: 0.97, 1.15; I2 = 56.2%; n = 6); pancreatic cancer, 1.16 (95% CI: 1.05, 1.28; I2 = 61.6%; n = 2); ovarian cancer, 1.08 (95% CI: 1.03, 1.13; I2 = 0%; n = 2); esophageal squamous cell carcinoma, 1.24 (95% CI: 1.10, 1.38; I2 = 64.3%; n = 2); renal cell carcinoma, 1.08 (95% CI: 1.02, 1.13; I2 = 0%; n = 2); and esophageal adenocarcinoma, 1.26 (95% CI: 1.13, 1.39; I2 = 0%; n = 2). A nonlinear dose-response meta-analysis showed that, after a somewhat unchanged risk within initial scores of the DII, the risk of colorectal cancer increased linearly with increasing DII score. In the analyses of breast and prostate cancers, the risk increased with a very slight trend with increasing DII score. In conclusion, the results showed that dietary habits with high inflammatory features might increase the risk of site-specific cancers.