High-viscosity modified asphalt binder (HVMA) is used widely as a polymer-modified binder in porous asphalt pavement because it can improve the cohesiveness of the asphalt mixture. However, because of the high voidage in the mixture, HVMA is vulnerable to aging induced by temperature, oxygen, water, sunlight, and other climatic conditions, which degrades the performance of pavement. The properties of asphalt binder are affected adversely by the effects of hygrothermal environments in megathermal and rainy areas. Therefore, it is essential to study the aging characteristics of HVMA under the influence of hygrothermal environments to promote its application as a high-viscosity modifier. A hygrothermal cycle aging test (HCAT) was designed to simulate the aging of HVMA when rainwater was kept inside of the pavement after rainfall in megathermal areas. One kind of base bitumen and three kinds of HVMA (referred to as SBS, A, and B, respectively) were selected in this study. Short-term aging tests, hygrothermal cycling aging tests, and long-term aging tests were performed on the base bitumen and three kinds of modified asphalt binder. Fourier-transform infrared spectroscopy (FTIR), thermo-gravimetric analysis (TGA), and dynamic shear rheological (DSR) tests were used to evaluate the properties of the binders on the micro and macro scales. By comparing the index variations of the four binders before and after aging, the effects of the hygrothermal environment on the properties of HVMA were studied. It was found that the effects of the hygrothermal environment expedited the decomposition of the polymer and the formation of carbonyl groups compared with the TFOT and PAV test, which TGA confirmed further. Moreover, the thermal stability of the samples was improved after HCAT. In addition, the master curves of the complex modulus showed that hygrothermal cycles made the high-temperature rutting resistance of asphalt binder increase significantly. All of the results above verified that the effect of hygrothermal cycling could accelerate the aging of HVMA and shorten its service life.