To reduce the temperature of asphalt pavement and improve the antirutting performance of asphalt mixture, a thermal-resistant asphalt mixture (TRAM) was produced, in which a certain proportion of mineral aggregate was replaced by ceramic (CE) or floating beads (FB) featuring low thermal conductivity. Firstly, a parallel plate test was developed to test the thermal conductivity of asphalt mixture added with different thermal-resistant materials. Secondly, the illumination test system was designed to study the visual cooling effect of different TRAM by imitating the natural environment. Finally, the effect of different thermal-resistant materials on asphalt pavement performance was evaluated. The results show that the addition of thermal-resistant materials can reduce the thermal conductivity and the temperature of asphalt mixture. The cooling effect of CE75 and CE100 (coarse aggregate substituted by 75% and 100% CE, respectively) is superior to other aggregates. The temperature reduction rates of CE75 and CE100 reache 6.6°C and 6.8°C, respectively. For FB50 and FB75 (fine aggregate substituted with 50 and 75% FB, respectively), the cooling effect of them reaches 3.9°C and 4.5°C, respectively. In addition, the CE and FB can improve the antirutting performance of asphalt mixture by reducing the temperature inside the pavement. The high-temperature performance of CE75 and FB75 is the best. With the increase of thermal resistance materials, the low-temperature cracking resistance of asphalt mixture decreases gradually. The failure strain of mixture added with 100% thermal resistance materials is close to the lower limit of Chinese specification. The water stability of different TRAM changes with various test methods. Taking into account the results of pavement performance and the cooling effect, the substitution proportion of CE and FB for TRAM is proposed as 50%∼75%, respectively.