In order to investigate the effect of Mg addition on the embrittlement of Cr-Mo steels, the 2.25Cr1Mo steel plates containing Mg, P contents were refined with vacuum induction furnace and rolled with double-stick reversible rolling mill. The impact toughness evolution and microstructural characteristics of these steels after aging at 580 °C for up to 5000 h were systematically investigated. The grain boundary segregation behaviors of P, S, and Mg before and after aging were analyzed with auger electron spectroscopy (AES), and the microstructure characteristics of the steels were detected with optical electron microscope (OM) and transmission electron microscope (TEM). The research results show Mg addition can improve the impact toughness of the 2.25Cr1Mo steel to a certain extent even with 0.056 wt.% P doping. It was clarified that Mg can segregate to grain boundary during the aging process, and its strong segregation tendency can reduce the grain boundary segregation of P to some degree. The effects of Mg on the impact toughness after subjecting to 580 °C ageing, including element segregation behaviors at grain boundary, ferrite formation, prior austenite grain characteristics, and carbides at grain boundary were also identified and discussed.