Fiber-reinforced polymer and metal laminated structures are widely used in aerospace and military fields because of their excellent mechanical and physical properties. However, the materials are prone to processing defects such as burring and furry during processing. In this paper, based on the mechanism of vibration cutting, the mixed-frequency vibration tool holder (MFVT) with low-frequency and ultrasonic combined vibration is designed. The low-frequency vibration module and ultrasonic vibration module of the tool holder are designed theoretically and studied experimentally. The vibration surface model and the dynamic equation of the low-frequency vibration module are established, which lays a theoretical foundation for the design of the low-frequency vibration module. A dynamic simulation model for error analysis is established, and the effects of surface profile error, cage clearance error and spline coupling clearance error on low-frequency vibration motion curve are studied. Among them, the contour error has the greatest influence on the motion curve. Analyzing the thermal energy generation mechanism of an ultrasonic vibrator through the vibration self-heating simulation model. The impact of temperature on the operational performance of an ultrasonic vibrator was analyzed through experiments. The experimental results indicate that both the amplitude and resonant frequency of the ultrasonic vibrator decrease with the increase in temperature.