Lung cancer is one of the most common cancers and the leading cause of cancer-related deaths worldwide. Most of these patients with non-small cell lung cancer (NSCLC) present with the advanced stage of the disease at the time of diagnosis, and thus decrease the 5-year survival rate to about 5%. Immune checkpoint inhibitors (ICIs) can act on the inhibitory pathway of cancer immune response, thereby restoring and maintaining anti-tumor immunity. There are already ICIs targeting different pathways, including the programmed cell death 1 (PD-1), programmed cell death ligand 1 (PD-L1), and cytotoxic T lymphocyte antigen 4 (CTLA-4) pathway. Since March 2015, the US Food and Drug Administration (FDA) approved nivolumab (anti-PD-1 antibody) as the second-line option for treatment of patients with advanced squamous NSCLC. Additionally, a series of inhibitors related to PD-1/PD-L1 immune-checkpoints have helped in the immunotherapy of NSCLC patients, and modified the original treatment model. However, controversies remain regarding the use of ICIs in a subgroup with targeted oncogene mutations is a problem that we need to solve. On the other hand, there are continuous efforts to find biomarkers that effectively predict the response of ICIs to screen suitable populations. In this review, we have reviewed the history of the continuous developments in cancer immunotherapy, summarized the mechanism of action of the immune-checkpoint pathways. Finally, based on the results of the first-line recent trials, we propose a potential first-line immunotherapeutic strategy for the treatment of the patients with NSCLC.