Using solid waste to replace limestone filler in asphalt concrete can not only reduce the cost of road construction, but also improve the utilization rate of solid waste. In this study, PHC pile waste concrete (PPWC) was innovatively used to replace limestone filler in asphalt mixture and its effect on the physical and rheological properties of asphalt mastics was studied. Firstly, PPWC was ground into filler particles with a diameter less than 0.075 mm. The physical properties, particle characteristics and chemical composition of PPWC filler and limestone filler were compared. Asphalt mastics were prepared with different filler-asphalt volume ratios (20%, 30% and 40%) and the physical properties, high-temperature rheological properties and low-temperature cracking resistance of asphalt mastics were tested. The experimental results showed that the surface of PPWC filler is rougher and has lower density and smaller particle size than limestone filler. When the filler content is the same, PPWC filler asphalt mastics have lower penetration and ductility, higher softening point than limestone filler asphalt mastics, and the viscosity of PPWC filler asphalt mastics is more sensitive than limestone filler asphalt mastics. PPWC filler asphalt mastics demonstrated superior high-temperature stability, but poorer low-temperature cracking resistance compared to limestone filler asphalt mastics. In conclusion, PPWC fillers can be used to replace limestone fillers in asphalt mixtures. The finding of this study will provide a new solution for the construction of eco-friendly roads.