Lung adenocarcinoma (LUAD) needs to be stratified for its heterogeneity. Oncogenic driver alterations such as EGFR mutation, ALK translocation, ROS1 translocation, and BRAF mutation predict response to treatment for LUAD. Since oncogenic driver alterations may modulate immune response in tumor microenvironment that may influence prognosis in LUAD, the effects of EGFR, ALK, ROS1, and BRAF alterations on tumor microenvironment remain unclear. Immune-related prognostic model associated with oncogenic driver alterations is needed. In this study, we performed the Cox-proportional Hazards Analysis based on the L1-penalized (LASSO) Analysis to establish an immune-related prognostic model (IPM) in stage I-II LUAD patients, which was based on 3 immune-related genes (PDE4B, RIPK2, and IFITM1) significantly enriched in patients without EGFR, ALK, ROS1, and BRAF alterations in The Cancer Genome Atlas (TCGA) database. Then, patients were categorized into high-risk and low-risk groups individually according to the IPM defined risk score. The predicting ability of the IPM was validated in GSE31210 and GSE26939 downloaded from the Gene Expression Omnibus (GEO) database. High-risk was significantly associated with lower overall survival (OS) rates in 3 independent stage I-II LUAD cohorts (all P < 0.05). Moreover, the IPM defined risk independently predicted OS for patients in TCGA stage I-II LUAD cohort (P = 0.011). High-risk group had significantly higher proportions of macrophages M1 and activated mast cells but lower proportions of memory B cells, resting CD4 memory T cells and resting mast cells than low-risk group (all P < 0.05). In addition, the high-risk group had a significantly lower expression of CTLA-4, PDCD1, HAVCR2, and TIGIT than the low-risk group (all P < 0.05). In summary, we established a novel IPM that could provide new biomarkers for risk stratification of stage I-II LUAD patients.