With the acceleration of the construction of sponge cities in China, porous asphalt pavement (PA) is has been widely used. High-viscosity asphalt (HVA) is the core material in building PA because it has good rheology properties, which can provide good raveling and rutting resistance. However, due to the open-graded structure of PA, HVA was more susceptible to rapid aging, which significantly affects the durability of PA. To investigate the thermal aging effect on the rheological properties of self-modified HVA (SHVA), five types of asphalts were aged using a rolling thin film oven (RTFO) and pressure aging vessel (PAV). Then, rheological tests were adopted, such as temperature sweep test (TS), repeated creep and recovery test (RCR), and bending beam rheometer test (BBR). The results indicate that during the aging process, the oxidation-induced hardening effect of neat asphalt and the degradation-induced softening effect of the modifier changes the rheology properties of HVA significantly. As the aging progresses, the contribution of the modifiers of HVA to anti-aging performance is greatly reduced. At high temperatures, HVA demonstrates better anti-aging performance than conventional styrene–butadiene–styrene (SBS)-modified asphalt (Guo Chuang, GC). The change of the high-temperature rheological indices of the two HVA types (SHVA and TAFPACK-super HVA (TPS)) showed a smaller activation energy index (EAI), a more considerable viscous component of binder creep stiffness (Gv), and more minor accumulated stain (racc), indicating a more significant anti-short-term and long-term aging performance, which is beneficial to the high-temperature performance of asphalts. However, the changes in low-temperature rheological properties do not align with those in high-temperature rheological properties after long-term aging. The BBR test results reveal that TPS exhibits worse low-temperature performance than GC and SHVA. During the thermal aging process, the contribution rate of the modifiers in SHVA against RTFO and PAV aging is higher than that of the modifiers in TPS, which contributes to the superior anti-aging property. Overall, SHVA demonstrates the best anti-aging performance among the five asphalts tested.