To accurately evaluate the high-temperature stability of cast asphalt concrete, based on the traditional viscoelastic mechanical model, this paper innovatively improves the traditional viscoelastic mechanical model considering the conditions of temperature and dynamic load and deduces the viscoelastic constitutive model of high-temperature deformation performance of cast asphalt concrete, using a dynamic penetration test (DPT) to regress improved the Burgers model parameters and verify the model’s applicability and reliability. The results show that for the modified Burgers model parameters, E1, E2, A, and η2 decrease with increasing temperature, while parameter B increases with the increase of temperature. The high-temperature deformation of cast asphalt concrete is mainly related to η1, and η1 decreases, and deformation increases with the increase in temperature. With the increase in loading times, the dynamic penetration of cast asphalt concrete (AC) gradually increases, that is, when the greater the permanent deformation is. When the specimen is loaded 3000 times, the dynamic penetration value at 60°C is about 1.237 mm, and the dynamic penetration value at 50°C is about 0.819 mm. Under other loading times, the permanent deformation of the specimen at 60°C is much greater than that at 50°C. There is a good fit between the mechanical model derived from the viscoelastic theory and the measured test data, and the correlation coefficient is greater than 0.98, indicating that the model can accurately describe the permanent deformation law of GA under periodic semivector sinusoidal pulse load at high temperatures. The research results of this paper provide a solid theoretical basis for the promotion and application of cast asphalt concrete, and the method has wide applicability in practical engineering.