Fly ash (FA) is characterized by its porous structure and richness in silicon and aluminum oxides; thus, it can be used as an adsorbent for heavy metals. In order to enhance the absorption efficiency and stabilization effect, we prepared a new fly ash (FAKCa) using calcium hydroxide (Ca(OH)2) and phosphate (KH2PO4) through a simple one-step low-temperature alkali dissolution method and investigated its adsorption performance for lead and cadmium in water solutions and the stabilization effects of lead and cadmium in soils under flooding condition. Results showed that the Langmuir model best fit the adsorption behavior of lead and cadmium, and the maximal adsorption capabilities of lead (128 mg/g) and cadmium (39.1 mg/g) for FAKCa were increased by 236% and 14.5% compared with the unmodified FA, respectively. The adsorption of lead and cadmium by FAKCa was better fitted to the second-order kinetic model. The enhancement of adsorption capacities for lead and cadmium may be partly due to the specific surface area of FAKCa, which was increased by 94.0% compared to unmodified FA. FTIR, XRD, and XPS analysis showed that the Si-O and Al-O functional groups, carbonate, and hydroxide precipitation were facilitated by the adsorption of lead and cadmium. Thus, ion exchange, surface complexation, and formation of metal hydroxide and carbonate precipitation were the main adsorption mechanisms for lead and cadmium by FAKCa. In addition, the application of 0.1–0.6% FAKCa increased soil pH by 0.19–0.67 units and decreased the CaCl2-extractable lead by 12.3–86.5% compared to FA. Meanwhile, FAKCa was more effective in transforming lead and cadmium from exchangeable to stable fractions. This study shows that calcium hydroxide–phosphate-modified fly ash could effectively increase the adsorption and stabilization of lead and cadmium and, thus, has great potential for large-scale applications in contaminated soil.