This paper attempts to analyze the effect of mineral powder on the adhesion of aggregates and asphalt. First, it employs the gravimetric method to quantitatively analyze the peeling rate of asphalt and asphalt mortar from the surface of basalt, andesite, and limestone aggregates. Then, it takes microscopic pictures of the mineral powder adhered on the surface of the aggregates to observe the distribution of mineral powder in the asphalt mixtures, and uses profiles to analyze the adhesion status of mineral powder on the surface of the aggregates. At last, this paper analyzes the effect of mineral powder particles on the surface/interface of the aggregates and asphalt, and the results indicate that mineral powder has increased the roughness of the surface of the aggregates, therefore it indirectly promotes the adhesion of asphalt on the surface of the aggregates; the mineral powder particles interact with the surface texture of the aggregates, forming an embedding and wedging structure in the texture; compared with aggregates with smooth surface, the adhesion promotion effect is stronger.