Sulfate attack is one of the most important problems affecting concrete structures, especially magnesium sulfate attack. This paper presents an investigation on the mechanical properties and damage evolution of high performance concrete (HPC) with different contents of fly ash exposure to magnesium sulfate environment. The microstructure, porosity, mass loss, dimensional variation, compressive strength, and splitting tensile strength of HPC were investigated at various erosion times up to 392 days. The ultrasonic pulse velocity (UPV) propagation in HPC at different erosion time was determined by using ultrasonic testing technique. A relationship between damage and UPV of HPC was derived according to damage mechanics, and a correlation between the damage of HPC and erosion time was obtained eventually. The results indicated that (1) the average increasing amplitude of porosity for HPCs was 34.01% before and after exposure to magnesium sulfate solution; (2) the damage evolution of HPCs under sulfate attack could be described by an exponential fitting; (3) HPC containing 20% fly ash had the strongest resistance to magnesium sulfate attack.