Background. Esophageal cancer (EC) is the 7th most common neoplasm and the 6th most common cause of cancer-related death worldwide. Immunotherapy is an effective treatment for EC patients. However, there are no dependable markers for predicting prognosis and immunotherapy responses in EC. Our study aims to explore prognostic models and markers in EC as well as predictors for immunotherapy. Methods. The expression profiles of EC were obtained from The Cancer Genome Atlas (TCGA), the Gene Expression Omnibus (GEO), and International Cancer Genome Consortium (ICGC) databases. Cox regression analysis was performed to construct a prognostic model. Overall survival and receiver operating characteristic curve analyses were applied to verify the accuracy of the model. The CIBERSORT algorithm was conducted to quantify the infiltration of different immune cells, and EC was grouped into three immune cell infiltration (ICI) clusters. PD-1 and PD-L1 expressions were compared between the ICI clusters. Overall survival analysis between ICI score and tumor mutation burden was conducted. The immunotherapy response of patients in different ICI score clusters was also compared. The copy number variations, somatic mutations, and single nucleotide polymorphisms were analyzed. Enrichment analyses were also performed. Results. A prognostic model was successfully constructed. Three ICI clusters were identified, and the clusters with high immune and stromal scores tended to have more immune-activated phenotypes and higher expressions of PD-1 and PDL1. The ICI score may be used as a predictor independent of tumor mutation burden. Patients with higher ICI score tended to have better immunotherapeutic responses than those with lower scores. Enrichment analyses showed that the differentially expressed genes were mostly enriched in microvillus and the KRAS and IL6/JAK/STAT3 pathways. The top eight genes with the highest mutation frequencies in EC were identified and all related to the prognosis of EC patients. Conclusions. Our study established an effective prognostic model and identified markers for predicting the prognosis and immunotherapy response of EC patients.