Rechargeable magnesium batteries (RMBs), which have attracted tremendous attention in large‐scale energy storage applications beyond lithium ion batteries, have many advantages such as high volumetric capacity, low cost, and environmental friendliness. However, the strong polarization effect, slow kinetic de‐intercalation of Mg2+ ions, and the incompatibility between electrodes and electrolytes limit their commercial application. Thus, developing stable and high‐efficiency electrode materials and optimization of electrolytes are key to promoting the practical application of RMBs. In this review, a summary and discussion are provided regarding the recent progress in the development of the key materials for RMBs, including cathodes, anodes, and electrolytes. The cathode materials including intercalation type cathodes and conversion type cathodes are classified and introduced in detail by the reaction mechanism, the effects of structure on the kinetics of Mg2+ ion migration are clarified; the modification and interface issues of Mg anode materials are comprehensively stated, and the potential development prospects of RMB electrolytes are systematically analyzed. In addition, the main opportunities and challenges in this field are briefly elaborated and discussed. Finally, this review will provide a framework for the key materials for RMBs as a reference for future research.