This paper establishes a five degree-of-freedom nonlinear dynamic model of the linear guideway considering the carriage flexibility, which is used to analyze the ball-groove load distribution and nonlinear dynamic characteristics under the action of the complex external loads. Firstly, according to the coordinate transformation matrix, the position relation between the groove centers when the carriage is a rigid body is determined. Next, the FE simulation is used to calculate the carriage flexible deformation caused by the contact loads, and the relation curves between the contact load and the displacement of the carriage groove center are determined. Then, the final position of the groove centers considering the carriage flexibility is determined, a 5 degree-offreedom nonlinear dynamic model of the linear guideway is established. Finally, considering the possibility of the ball-groove losing contact, the Newmark-Newton numerical algorithm is established to solve the proposed dynamic model, and the proposed model is validated through experiments. Furthermore, the nonlinear static and dynamic characteristics of the linear guideway under the effect of the carriage flexibility are studied, and the effects of the design parameters on the vibration characteristics are discussed.