The rolling element skidding may lead to the failure of the rolling bearing. The skidding characteristics can be effectively analyzed by using dynamic response of the rolling bearing. A dynamic model is established to investigate the vibration response of the rolling bearing with local defect on inner/outer race in this paper. In the proposed model, the rolling element skidding, contact stiffness and displacement, the interaction force between ball and race, the interaction force between cage and race, elastohydrodynamic lubrication are taken into consideration. The dynamic responses of the rolling bearing with the rolling element skidding are solved by the proposed model in the time and frequency domains. The effects of defect size, rotational speed, external load, and compound factors on skidding characteristics are investigated. The proposed model is verified by the experiments. The results show that the rolling element skidding leads to the significant difference of dynamic characteristics in the time and frequency domains, which aggravates the failure of the rolling bearings.