Nickel-based single crystal alloys are widely used in aerospace and other important fields of national defense due to their excellent properties. Phase transformation occurs during high-speed cutting of nickel-based single crystal alloy, which seriously affects the surface quality. It is of great significance to carry out theoretical research on phase transformation for improving the machining quality of nickel-based alloy. In this paper, molecular dynamics method is used to study the nano-cutting of single crystal nickel-based alloy with silicon nitride ceramic tool. The mechanism of phase transformation and the effect of cutting speed on phase transformation in workpieces are studied in detail. The nano-cutting model is established. Morse potential functions for molecular dynamics simulation are calculated, and EAM and Tersoff potential functions are selected. The effect of cutting speed on phase transformation was studied by using radial distribution function, coordination number analysis, common neighbor analysis, and the deep reasons for the sharp change of lattice structure were analyzed from many aspects. Finally, in order to verify the universality of the research results and explore the new properties of compression, nano compression (the same strain rate as the nano cutting process) was simulated. The results show that the increase of cutting speed leads to the increase of hydrostatic stress, the increase of energy in crystal and the rise of cutting temperature. As a result, the change of lattice structure becomes more and more intense, and the conversion rate of different crystal structures increases greatly.