The present study envisages the study of the residual stress characteristics of titanium alloy surface, based on the ultrasonic longitudinal–torsional composite milling process. Initially, the tool path analysis of common milling and ultrasonic longitudinal–torsional composite milling was performed. When compared with the common milling, it was found that the tool–chip cycle separation phenomenon occurred in the torsional vibration direction in the ultrasonic milling tool path. Subsequently, an ultrasonic longitudinal–torsional composite milling experimental platform was built and the effects of the ultrasonic amplitude (A), cutting speed ( vf), axial depth of cut ( ap), and feed per tooth ( fz) on the residual stress of titanium alloy were studied using an orthogonal experiment. The results indicated that the residual stress in ultrasonic longitudinal–torsional composite milling comprised a combination of the “plastic bulge effect” caused by cutting force–cutting heat and the “extrusion effect” of the tool flank on the workpiece. The “extrusion effect” was dominant and presented a compressive stress on the machined surface. The ultrasonic amplitude (A) showed the most significant influence on the residual stress. With an increase of the amplitude, the extrusion and collision effect of the tool on the workpiece surface became more obvious, along with a stronger “extrusion effect” between the tool and the workpiece surface. Experimental results provide a good processing method for the difficult-to-machine materials such as titanium alloy, and offers an effective method for obtaining large surface residual compressive stress.