Cold helical rolling (CHR) is one of the most effective ways to produce small-diameter steel balls. In this study, one kind of work hardening model was established and implemented into Simufact 15.0 to investigate the work hardening phenomenon in the cold forming process. Firstly, based on the helical rolling theory, a set of finite element (FE) simulations was developed. The influence of CHR parameters, including the starting height of convex rib, forming area length, and rolling inclination angle, on the forming process was studied via simulation. Furtherly, the CHR process experiments and FE simulation were carried out , the results showed that the FE simulation was in good agreement with the experimental results, and consistent with the predicted value of the theoretical calculation. Finally, the evolution of effective strain, effective stress, rolling force, work hardening and microstructure during the cold helical rolling of Φ 5.12 mm steel balls was investigated via FE. As result, the evolution trend of hardness was consistent with that of dislocation density, indicating that the model is credible. Besides, the microstructure of the steel ball at different positions further verified this.