The compaction of asphalt pavement is a crucial step to ensure its service life. Although intelligent compaction technology can monitor compaction quality in real time, its application to individual asphalt surface courses still faces limitations. Therefore, it is necessary to study the compaction mechanism of asphalt pavements from the particle level to optimize intelligent compaction technology. This study constructed an asphalt pavement compaction model using the Discrete Element Method (DEM). First, the changes in pavement smoothness during the compaction process were analyzed. Second, the changes in the angular velocity of the mixture and the triaxial angular velocity (TAV) of the mortar, aggregates, and mixture during vibratory compaction were examined. Finally, the correlations between the TAV amplitude and the coordination number (CN) amplitude with the compaction degree of the mixture were investigated. This study found that vibratory compaction can significantly reduce asymmetric wave deformation, improving pavement smoothness. The mixture primarily rotates in the vertical plane during the first six passes of vibratory compaction and within the horizontal plane during the seventh pass. Additionally, TAV reveals the three-dimensional dynamic rotation characteristics of the particles, and the linear relationship between its amplitude and the pavement compaction degree aids in controlling the compaction quality of asphalt pavements. Finally, the linear relationship between CN amplitude and pavement compaction degree can predict the stability of the aggregate structure. This study significantly enhances quality control in pavement compaction and advances intelligent compaction technology development.