The role of inflammation in neuroendocrine tumors is poorly known. The purpose of this study was to characterize the densities of CD3+, CD8+, CD4+ and FOXP3+ T cells in small bowel neuroendocrine tumors (SB-NETs), SB-NET lymph node metastases and gastric neuroendocrine tumors (G-NETs) to assess the prognostic role of immune cell infiltrates in SB-NETs. The final cohort included 113 SB-NETs, 75 SB-NET lymph node metastases and 19 G-NETs from two Finnish hospitals. CD3+- and CD8+-based immune cell score (ICS), and other T cell densities were evaluated. Survival analyses of SB-NETs and SB-NET lymph node metastases were performed with the Kaplan-Meier method and Cox regression adjusted for confounders. The primary outcome was disease-specific survival (DSS). No significant difference in DSS was seen between low and high ICS groups in SB-NETs at 5 years (92.6% vs. 87.8%) or 10 years (53.8% vs. 79.4%), p = 0.507, or in SB-NET lymph node metastases at 5 years (88.9% vs. 90.4%) or 10 years (71.1% vs. 59.8%), p = 0.466. Individual densities of the examined T cell types showed no correlation with prognosis either. SB-NETs and lymph node metastases had similar inflammatory cell profiles, whereas in G-NETs CD3+ and CD8+ T cells were particularly more abundant. In SB-NETs, ICS or T cell densities showed no correlation with prognosis.