Background: Small cell carcinoma of the esophagus is a rare malignant tumor. We aimed to explore the chemotherapeutic efficacy on the prognosis of patients with small cell carcinoma of the esophagus who received radiotherapy. Methods: To identify the population of interest, Surveillance, Epidemiology, and End Results data from 1996 to 2016 were chosen. Univariate and multivariate analyses were used to probe into prognosis factors. Multivariate Cox regression analysis was conducted to identify factors related to overall survival and cancer-specific survival. Results: Overall, data from 162 patients were analyzed in this study. Tumor size (P = 0.014), T staging (P = 0.028), and chemotherapy (P < 0.001) were independent prognostic factors affecting overall survival. Patients with regional disease (hazard ratio = 5.435, P < 0.001) and distant metastasis (hazard ratio = 2.183, P < 0.001) who received radiotherapy alone had worse survival than those receiving chemoradiotherapy. Tumor size (P = 0.004) and chemotherapy (P < 0.001) were independent prognostic factors affecting cancer-specific survival. Tumor size was an independent factor affecting cancer-specific survival for patients receiving chemoradiation. Conclusions: Age, T staging, tumor size, primary site, and chemotherapy are independent prognosis factors affecting overall survival and cancer-specific survival in patients with small cell carcinoma of the esophagus who receive radiotherapy. Chemotherapy might further improve cancer-specific survival in patients with small cell carcinoma of the esophagus receiving radiotherapy at all stages.