Objective: The goal of surveillance after therapy of localized esophageal cancer (LEC) is to identify actionable relapses amenable to salvage; however, the current surveillance algorithms are not optimized. We report on a large cohort of LEC patients with actionable locoregional relapses (LRRs). Methods: Between 2000 and 2013, 127 (denominator = 752) patients with actionable LRR were identified. Histologic/cytologic confirmation was the gold standard. All surveillance tools (imaging, endoscopy, fine needle aspiration) were assessed. Results: Most patients were men (89%), had adenocarcinoma (79%), and had no new symptoms (72%) when diagnosed with LRR. In trimodality patients, endoscopic confirmation of positron emission tomography-computed tomography-suspected LRR occurred in only 44%, and 56% required additional tools (e.g., fine needle aspiration). Alternatively, in bimodality patients, endoscopy confirmed LRRs in 81%. Trimodality patients had a higher risk of subsequent LRR/distant metastases after the first LRR than the bimodality patients (p = 0.03). In all patients, 78% of the subsequent relapses were distant. For patients who were salvaged, survival was significantly prolonged (50.6 vs. 25.1 months, p < 0.01). Conclusions: Patients live longer after successful salvage of the LRR than if salvage is not possible. After LRR, patients have a high risk of subsequent distant metastasis and whether the second relapse is local or distant, survival is uniformly poor.