BACKGROUND The red mud (bauxite residue) is a rare earth element (REE) bearing material that should be considered a valuable resource. This research investigated the selective leaching of scandium (Sc) and yttrium (Y) from red mud by agitation leaching, as well as thermal and hydrothermal processing. Material characterization was conducted to shed light on the leaching mechanism for high REE extraction selectivity. RESULTS The agitation leaching with HCl, HNO3, and H2SO4 was insufficient for high leaching efficiencies of REEs. The addition of Ethylenediaminetetraacetic acid (EDTA) had a minor influence in the leaching of elements. Two acid combinations (0.27 Μ HCl + 0.2 Μ H2SO4 and 0.27 Μ HCl + 0.2 Μ HNO3) were formulated to optimize the leaching efficiency of REEs under thermal and hydrothermal processing. Thermal processing showed high Y leaching, but the selectivity was low because of considerable Ti and Fe extraction. High REE leaching selectivity was achieved when HCl + HNO3 was used in hydrothermal processing at 180 °C. Material characterization indicated that dense aggregation of particles occurred after hydrothermal processing. The mineral geothite was dissolved and new crystals like potassium calcium silicate were formed. CONCLUSION The high selective leaching of Sc and Y during hydrothermal processing was controlled by the extraction and desorption of elements from the material surface. The precipitation of Fe, Ca, and Ti and their re‐crystallization in new minerals have enhanced leaching selectivity. © 2021 Society of Chemical Industry (SCI).