No clinically relevant biomarker has been identified for predicting the response of esophageal squamous cell carcinoma (ESCC) to chemoradiotherapy (CRT). Herein, we established a CT-based radiomics model with artificial intelligence (AI) to predict the response and prognosis of CRT in ESCC. A total of 44 ESCC patients (stage I-IV) were enrolled in this study; training (n = 27) and validation (n = 17) cohorts. First, we extracted a total of 476 radiomics features from three-dimensional CT images of cancer lesions in training cohort, selected 110 features associated with the CRT response by ROC analysis (AUC ≥ 0.7) and identified 12 independent features, excluding correlated features by Pearson’s correlation analysis (r ≥ 0.7). Based on the 12 features, we constructed 5 prediction models of different machine learning algorithms (Random Forest (RF), Ridge Regression, Naive Bayes, Support Vector Machine, and Artificial Neural Network models). Among those, the RF model showed the highest AUC in the training cohort (0.99, p < 0.001) as well as in the validation cohort (0.92, p < 0.001) to predict the CRT response. Additionally, Kaplan-Meyer analysis of the validation cohort and all the patient data revealed that the PFS and OS in the high-prediction score group were significantly longer than those in the low-prediction score group. Univariate and multivariate analyses revealed that the radiomics prediction score could be an independent prognostic biomarker, and moreover significantly superior to serum SCC-Ag, the conventional tumor marker of ESCC. In conclusion, we have developed a novel and robust CT-based radiomics model using AI, which successfully predicts the CRT response as well as the prognosis for ESCC patients with high accuracy, non-invasiveness, and cost-effectiveness.