Background: Esophageal squamous cell carcinoma (ESCC) is the most prevalent histological type of esophageal cancer, but there is a lack of definite prognostic markers for this cancer.Methods: We used the ESTIMATE algorithm to access the tumor microenvironment (TME) of ESCC cases deposited in the TCGA database, and identified TME-related prognostic genes using Cox regression analysis. A least absolute shrinkage and selector operation or LASSO algorithm was used to identify key prognostic genes. Risk scores were calculated, and a clinical predictive model was constructed to evaluate the prognostic value of TME-related genes.Results: We found that high immune and stromal scores were significantly associated with poor overall survival (p < 0.05). We identified a total of 1,151 TME-related differently expression genes, among which 67 were prognosis-related genes. Through the LASSO method, 13 key prognostic genes were selected, namely, ADAMTS16, LOC51089, CH25H, CORO2B, DLGAP1, GYS2, HAL, MXRA8, NPTX1, OTX1, RET, SLC24A2, and SPI1, and a 13-gene risk score was constructed. A higher score was indicative of a poorer prognosis than a lower risk score (hazard ratio = 8.21, 95% confidence interval: 2.56–26.31; P < 0.001). The risk score was significantly correlated with immune/stromal scores and various types of infiltrating immune cells, including CD8 cells, regulatory T cells, and resting macrophages.Conclusion: We characterized the tumor microenvironment in ESCC, and identified the key prognosis genes. The risk score based on the expression profiles of these genes is proposed as an indicator of TME status and is instrumental in predicting patient prognosis.