In this paper, a nine degree-of-freedom dynamic model of the ball screw feed system considering the contact nonlinearity between balls and raceways is established to analyze the vibration characteristics. The position relationship between raceway centers for the ball screw and bearings is determined by using the homogeneous coordinate transformation, and then the restoring force functions along the axial and lateral directions are derived. The dynamic equations of the feed system are solved by using Newmark method, and the proposed model is verified by the experimental method. Furthermore, the effect of the excitation amplitude on the axial vibration of the feed system is investigated by the frequency-amplitude curve and 3-D frequency spectrum. With the increase of excitation amplitude, the dynamic response of the feed system exits the softening, hardening type nonlinearity and jump phenomenon. Additionally, the effects of the initial contact angle, length of screw shaft and number of loaded balls on the axial vibration of the feed system in the resonance region are discussed. The results show that the dynamic model established in this paper is suitable for improving the machining accuracy and stability of the ball screw feed system.