Conventional carburizing has disadvantages, such as high energy consumption, large deformation of parts, and an imperfect structure of the carburizing layer. Hence, a rare earth ion pre-implantation method was used to catalyze and strengthen the carburized layer of 20Cr2Ni4A alloy steel. In this study, X-ray diffraction (XRD), X-ray photoelectron spectroscopy (XPS), optical microscopy (OM), scanning electron microscopy (SEM), energy dispersive microanalysis (EDS), transmission electron microscopy (TEM), and Rockwell/Vickers hardness testing were used to analyze the microstructure, phase composition, retained austenite content, hardness, carburized layer thickness, and carbon diffusion. The results showed that lanthanum and yttrium ions implanted into the 20Cr2Ni4A steel formed solid solutions of rare earth ions and a large number of dislocations, which improved the diffusion coefficient of carbon elements on the carburized surface and the uniformity of the carbon distribution. Simultaneously, rare earth ion implantation improved the structure and hardness of the vacuum carburized layer. Compared to the lanthanum ion implantation, yttrium ion implantation caused the structure of the carburized layer to be finer, and the carbon diffusion coefficient increased by 1.17 times; in addition, the surface hardness of the carburized layer was 61.8 HRC.