Asphalt mixture is a temperature sensitive material. With the change of external environment temperature, asphalt pavement is prone to temperature-related diseases. Adding phase change material (PCM) to asphalt pavement to adjust pavement temperature is one of the effective methods at present. At present, there are many types of research on PCMs, but the research on PCMs added to the pavement structure is very scarce. In this paper, through the temperature test of rutting plate specimens with different layer combinations, the cooling effect of pavement structure combinations with PCMs added to different layers (upper layer, middle layer, and bottom layer) in pavement structure under different illumination times are discussed. Through the self-designed environmental simulation box, the real-time monitoring of the temperature of different layers in the pavement structure is realized. The cooling effect between different layers in different phase change pavement structure combinations is analyzed, and compared with each layer of ordinary pavement structure, and the best addition method is obtained for phase change materials, which provides a certain reference for the construction and specific application of PCM asphalt pavement, and made important contributions to the development of asphalt cooling pavement. The results show that the PCM can effectively reduce the temperature of each layer of the pavement structure. Under different illumination durations, the cooling effect of the samples with PCM in the upper layer was the worst. The samples with phase change material in the middle layer had the best cooling effect in the upper and middle layers of the pavement. The addition of phase change material to both the upper and middle layers had the most obvious cooling effect on the lower layer of the pavement. Therefore, combined with the comprehensive consideration of economy and cooling effect, the comprehensive cooling effect of adding PCMs to the middle surface layer is the best. It is recommended to add phase change materials to the middle surface layer during asphalt cooling pavement construction.