The accurate prediction and research about the dynamic characteristic of machine tool during the design process are essential to develop high-performance machines. In order to consider the effect of mechanical modal characteristics on the whole dynamic performance and servo controller of ball screw feed drive fully, the modal characteristics of mechanical transmission are extracted and solved based on reduced state space matrix of ball screw feed system, which is also easy for integration with servo control model. On this basis, the step-by-step optimization method from the global to the local is proposed to optimize the servo controller and overall dynamic performance of ball screw feed drive. This step-by-step optimization method is as follows: first, the global initial optimal tuning of servo controller parameters under the influence of structure modal characteristics is originally realized through the analytic derivation of servo controller based on the extraction of state space modal characteristics. Second, the integrated model of the feed drive system is established. Finally, based on the above global optimization results, the overall dynamic performance, and the servo parameters of the feed drive system under the influence of the modal characteristics are further studied and optimized locally. Taking the ball screw feed drive studied in this paper like the example, its rise time and the settling time of step response are 0.089 and 0.104s before optimization, while these reduce to 0.038 and 0.092s after optimization. Its dynamic response performance after optimization is much better. The optimization result is also validated by the test: the rise time and the settling time by the test are 0.042 and 0.094s, which are close to that from the simulation. Also, the consistency of step response curves from experiment and simulation as well. All these show that: this prediction and optimization method of overall dynamic performance considering the modal characteristic influence of mechanical transmission fully is correct and reliable.INDEX TERMS Ball screw feed drive in machine tool, dynamic characteristic, servo parameters, modal property, state space, step-by-step optimization method from the global to the local.