The dynamic response of particles is closely related to screening efficiency. To study the dynamic response of particles, the dynamic equations of a particle on a screening surface are established based on the elastoplastic contact model of spherical particles and are solved for the coal particle. Then, the trajectories of the particles are given with different falling heights and particle radii. The completely different trajectories with slight changes in the falling height and particle radius indicate strong nonlinearity. Second, the nonlinear dynamic behavior under different amplitudes and frequencies is discussed, and the route of transition from quasiperiodic motion to chaotic motion is revealed. Finally, we discuss the average speed along the screening surface considering the frequency, amplitude, friction coefficient, inclination angle, and vibration direction angle. In addition, the convergence conditions of particle motion are proposed, and they are only affected by the inclination angle and friction angle. The results show that in the normal direction of the vibrating screen, the particle motion is quasiperiodic at low frequencies. With increasing frequency, the motion of the particle becomes chaotic, and its Poincaré map becomes petal-shaped. In addition, the number of petals increases at the mutation of the bifurcation diagram. The increase in frequency, amplitude and inclination angle and the decrease in friction coefficient lead to an increase in particle speed along the screen surface. In addition, the particle speed reaches a maximum when the vibration direction angle is 65°. This work provides a theoretical basis for controlling the thickness of granular material flow on a vibrating screen and selecting screening process parameters.