Abrasive Water Jet (AWJ) technology has vast application prospects in the assisted drill-blasting of tunnel excavation, with the advantages of fast-breaking speed, low tool wear, less dust, and good mobility. Nevertheless, AWJ technology has some limitations and shortcomings, such as the small effective fracturing range and parameter mismatch, which influence the fracturing effect of AWJ. To solve the abovementioned issues, it is necessary to study the failure mechanism of rock impacted by AWJ and nozzle parameter effects on rock fragmentation. Based on the coupling algorithm of Smooth Particle Hydrodynamics and Finite Element Method (SPH-FEM), in this research, the numerical model of AWJ impacting rock was established, and the result was verified with Computed Tomography (CT) scanning after the AWJ impacting rock experiment and image processing technology. Through the analysis of the stress characteristics of typical particles in the rock model at different stages and positions, the formation and expansion mechanisms of the crater and the cracks were revealed. Additionally, in this research, the comprehensive damage factor of rock (X) representing the fragmentation degree was defined. By comparatively analyzing X-values with certain technical parameters of AWJ, the importance ranking of the nozzle parameters, the effect of each nozzle parameter on the rock fragmentation, and the optimal parameter combination were also investigated.