The automatic cutting of coal and rock surface morphology modeling based on the actual geological environment of coal mine underground excavation and mining is of great significance for improving the surface quality of coal and rock after cutting and enhancing the safety and stability of advanced support. To this end, using the principle of coordinate transformation, the kinematic trajectory of the cutting head of the tunneling machine is established, and the contour morphology of the cutting head under variable cutting technology is obtained. Then, based on the regenerative vibration theory of the cutting head, a dynamic model of the cutting head coal wall is established, and the coordinate relationship of the cutting head in the tunnel coordinate system under vibration induction is analyzed. Based on fractal theory and Z-MAP method, a simulation method for the surface morphology of coal and rock after cutting is proposed, which is driven by the cutting trajectory Under the coupling effect of cutting vibration induction and random fragmentation of coal and rock, simulation of the surface morphology of comprehensive excavation tunnels was conducted, and relevant experiments were conducted to verify the results. A 1:3 similarity experimental model of EBZ160 tunneling machine was used to build a cutting head coal and rock system cutting experimental platform for comparative experiments of cutting morphology. Furthermore, statistical methods were used to compare and evaluate the simulated roof with the actual roof. The results show that the relative errors between the maximum range of peaks and valleys, the peak skewness coefficient of height standard deviation, and the kurtosis coefficient of the actual roof are 1.3%, 24.5%, 16%, and 2.9%, respectively. Overall, this indicates that the surface morphology distribution characteristics of the simulated roof and the actual roof are similar, verifying the effectiveness of the modeling and simulation method proposed in this paper, and providing theoretical support for the design and optimization of advanced support in the future.