The multi-domain modeling and simulation of the precision high-speed punching press was carried out using the software SimulationX in this article. The key technical problems of the multi-domain modeling, such as the establishment of the subsystems of mechanics with accurate structures and dimensions, improvement of modeling accuracy and simulation efficiency by dividing and dealing with components in group and import of the displacement-depended stamping forces on the stamping dies, have been resolved to establish the integral multi-domain model, which contains subsystems of mechanics, hydraulic and servo driven of the complete punching press. Vibration quantity of the punching press was predicted through simulation, and validity of the multi-domain model was confirmed by experimental measurement. The vibration reduction in the punching press based on the topology optimization of the slider and design of rotational speed variation curve of the servo motor were investigated. The results show that through topology optimization, the mass of the slider could be decreased under the premise of guaranteeing the structural strength, by which the vibration quantity of the punching press can be decreased; rotational speed of the servo motor could be decreased during the stamping period by designing the rotational speed variation curve, which leads to a decrease in acceleration and vibration of the slider during that period, with which the forming precision can be guaranteed.