In order to represent the mechanical response laws of high-modulus asphalt pavement (HMAP) faithfully and objectively, the viscoelasticity of high-modulus asphalt mixture (HMAM) was considered, and the viscoelastic mechanical responses were calculated systematically based on moving load by numerical simulations. The performances of the HMAP in resistance to the deformation and the cracking at the bottom layer were compared with the ordinary asphalt pavement. Firstly, Lubao and Honeywell 7686 (H7686) were selected as the high modulus modifiers. The laboratory investigations of Asphalt mix-70 penetration, Asphalt mix-SBS (styrene-butadiene-styrene), HMAM-Lubao and HMAM-H7686 were carried out by dynamic modulus tests and wheel tracking tests. The conventional performances related to the purpose of using the HMAM were indicated. The master curves of the storage moduli were obtained and the viscoelastic parameters were fitted based on viscoelastic theories. Secondly, 3D pavement models based on moving loads for the viscoelastic structures were built using the non-linear finite element software ABAQUS. The wheel path was discretized in time and space to apply the Haversine wave load, and then the mechanical responses of four kinds of asphalt pavement were calculated. Finally, the sensitivity analysis was carried out. The results showed that the addition of the high modulus modifiers can improve the resistance to high-temperature rutting of the pavements. Except for the tensile strain and stress at the bottom of the underlayer, other responses decreased with the increases of the dynamic moduli and the change laws of the tensile strain and stress were affected by the range of the dynamic modulus. The tensile stress at the bottom of the asphalt layer would be too large if the modulus of the layer were too large, and a larger tensile strain would result. Therefore, the range of the modulus must be restricted to avoid the cracking due to excessive tension when using the HMAM. The resistance of the HMAP to deformation was better and the HMAP was less sensitive to load changes and could better withstand the adverse effects inflicted by heavy loads.