Dam concrete suffers from serious abrasion damages in southwestern China, the abrasion resistance of concrete is therefore one of the most important factors determining the reliability even the safety of the dams. In the present work, the effects of fly and/or silica fume on the mechanical properties, drying shrinkage, as well the cracking and abrasion resistance of concrete were investigated, then the pore structures of concrete added with fly ash and silica fume and the pore surface fractal dimensions (Ds) were determined by the mercury intrusion porosimetry (MIP) method and a fractal model, respectively. Finally, the relationships between the abrasion resistance of concrete and the porosity as well as the Ds were revealed and discussed. The results indicate that silica fume significantly increases the drying shrinkage especially at early age, while the incorporation of fly ash and silica fume together can decease the early plastic shrinkage-induced cracking risk and the final shrinkage to some degrees. Besides, the utilization of 5 wt% silica fume and 20 wt% fly ash together increases the abrasion resistance and mechanical property by about 4-9 % at various ages. In addition, the compressive strength and the abrasion resistance of concrete are linearly correlated with the concrete porosity and the Ds. Both the fly ash and silica fume could decrease the porosity of concrete and increase the Ds, therefore the concrete containing fly ash