M50 bearing steel has great potential for applications in the field of aerospace engineering, as it exhibits outstanding mechanical and physical properties. From a microscopic point of view, bearing wear originates from the microscopic region of the contact interface, which usually only contains hundreds or even several atomic layers. However, the existing researches seldom study the wear of M50 bearing steel on the microscopic scale. This study explored the atomic-scale modeling method of M50 bearing steel. Then molecular dynamics simulations of nanoindentation on the M50 bearing steel model were carried out to study the size effect of the mechanical behaviors. The simulation results show that with the change in the radius of the diamond indenter in the nanoindentation simulation, the calculated nanohardness decreases. According to the size effect, when the indentation radius is 200 nm, the hardness obtained by the simulation is about 9.26 GPa, and that of the M50 sample measured by the nanoindentation is 10.4 GPa. Then nanoindentation simulations were carried out at different temperatures. The main bearings of aero-engines generally work at 300–500 degrees Celsius. When the simulated temperature was increased from 300 K to 800 K, the hardness of the model decreased by 15%, and the model was more prone to plastic deformation. In this study, a new molecular dynamics modeling method for M50 bearing steel was proposed, and then nanoindentation simulation was carried out, and the nanoindentation experiment verified the correctness of the model. These results are beneficial to the basic understanding of the mechanical performance of M50 bearing steel.