The carburized holes processed by ordinary internal grinding are prone to burn, crack, and low efficiency. Honing has a superior machining efficiency and cooling effect compared to traditional internal grinding. In this paper, we innovatively apply honing to carburizing hole grinding and propose an effective optimization scheme to enhance the surface finish of carburized holes. We set up an experimental system to explore the influence law of honing head rotation speed, axial reciprocating speed, grain size, and single grinding depth on surface roughness. Based on the grey correlation and response surface method, we propose a method to optimize the honing parameters of carburized holes and establish a prediction model, which has an R2 value of 0.9887, indicating that the model fits well. We verify the validity of the model by the root mean square error of 0.012 between the measured and calculated values. Based on the model, the optimal parameters of roughness (Ra) is obtained and verified by experiments. Compared with the original honing parameters, the surface roughness quality is improved by 25.8%. It shows that the optimized honing process based on the GRA-RSM method improves the surface quality of carburized holes significantly.