Immune checkpoint inhibitors (ICI), such as anti-PD-1 agents, have become part of the standard of care treatment of advanced non-small cell lung cancer (NSCLC). Predictive biomarkers are needed to identify patients that benefit from anti-PD-1 treatments. Tumor infiltrating lymphocytes (TILs) and PD-L1 are major players in the ICI mechanism of action. In this study, we assess the impact of real-world clinicopathological variables, including TILs and PD-L1, on anti-PD-1 efficacy. Methods: We performed a monocenter retrospective study in advanced NSCLC treated with nivolumab or pembrolizumab between January 2015 and February 2019. The impact of baseline clinical and pathological variables was assessed by univariate and multivariate models. TILs, defined as CD8+T-cells, and PD-L1 were scored in tumor and stroma, and correlated with progression free survival (PFS) and overall survival (OS). Results: We included 366 patients of whom 141 were assessed for tumor and stromal TILs. The median follow-up time was 487 days. In the whole cohort, PFS was associated with high tumor PD-L1, high albumin and good performance. OS was associated with low LDH, high albumin, good performance and 'first-line treatment'. In the TILs subcohort, stromal TILs had the strongest impact on PFS and OS. Stromal TILs were a stronger marker for PFS and OS than tumoral TILs, tumoral PD-L1 or stromal PD-L1. Remaining factors for PFS and OS were albumin and albumin with LDH, respectively. Conclusions: This real-world study on clinicopathological features shows that stromal CD8 + TILs were the strongest predictor for PFS and OS in patients with advanced NSCLC on anti-PD-1 therapy. Other predictors for PFS and OS included albumin and albumin together with LDH, respectively. This study highlights the pivotal role of the stromal compartment in the mechanisms of action of ICI, and the need for further studies aiming to overcome this stromal firewall.