Carbon fiber-reinforced composites (CFRP) are widely used in aerospace structural holes due to their superior mechanical properties. However, fiber pulling, delamination, burr and other defects often occur in the parts, which seriously affect the accuracy and service life of the products. The traditional cylindrical milling cutter is easy to lead to the exit delamination defects due to the large axial force at the bottom edge, concentrated wear at the periphery edge, and weak rigid constraint at the exit machining position. Therefore, this paper proposed a new hole machining strategy, analyzed the kinematic law of the tapered progressive spiral milling hole, studied the cutting mechanism of the four stages of the tapered spiral bidirectional milling hole, explored the material removal force state of the ultrasonic augment-assisted progressive milling hole, and established the prediction of the layered axial force of the ultrasonic progressive milling hole. Finally, combined with the material removal mechanism of ultrasonic vibration, the self-designed bidirectional progressive spiral milling cutter was used to verify the mechanism of material removal by progressive milling, in which the axial force could be effectively reduced. After 60 and 90 holes were processed, the axial force was about 18.5 and 23.3 N, respectively, the critical force of delamination could be improved, and the hole quality could be guaranteed without obvious burr and delamination phenomena.