(1) Background: Relapsed/refractory (r/r) and secondary acute myeloid leukemia are highlighted by chemoresistance and poor outcomes. The aim of the study was to assess the efficacy and toxicity of fludarabine, cytarabine, and granulocyte-colony stimulation factor (FLAG) with or without idarubicin (-Ida) and to discuss novel therapies in this setting. (2) Methods: Clinical and cytogenetic data on 130 consecutive patients with r/r and secondary AML treated at our center were retrospectively analyzed. (3) Results: There were 48, 56, and 26 patients with relapsed, refractory, and secondary AML, respectively. The median age was 60 years. The overall response was achieved in 70% of patients. The median overall survival (OS) time for the whole group was 9.4 months. In total, 47% of patients proceeded to allogeneic hematopoietic stem cell transplantation (aHSCT) and these patients had significantly prolonged OS compared to the others (63 months vs. 4.2 months; p < 0.001). Among the variables, including age, FLT3 mutation status, European LeukemiaNet (ELN) 2022 classification risk, FLAG vs. FLAG-Ida, and aHSCT, a multivariate analysis revealed that only aHSCT significantly influenced overall survival. (4) Conclusions: FLAG(-Ida) chemotherapy remains an effective salvage chemotherapy for patients with r/r and secondary AML with a plan of proceeding to aHSCT.