Previous studies have indicated that steel slag can be used as a substitute for natural aggregates in asphalt mixture, while little attention has been paid to systematic investigation of the influences of various external environmental factors on deformation resistance of steel slag asphalt mixture. In order to understand the service behavior of steel slag asphalt mixture, its permanent deformation under different condition was investigated based on an indoor simulation test. The chemical composition, microscopic structure, surface texture, and volume stability of steel slag were firstly characterized. The uniaxial repeated loading test and standard wheel-tracking test were applied to evaluate the effect of temperature, stress levels, and water damage on the permanent deformation of AC-16 and AC-20 steel slag asphalt mixtures. The results indicate that a higher content of alkaline oxide and high-grade texture index existing in steel slag contribute to its strong absorptivity and adhesion of asphalt. The steel slag demonstrates fine volume stability due to its lower free-CaO (f-CaO) content, autoclave chalked ratio, and immersion expansion ratio. The permanent deformation of steel slag asphalt mixtures increases rapidly under higher stress and temperatures in contrast to lower increment at lower stress and temperatures. Asphalt mixtures at higher stress and higher temperatures and water condition exhibit larger rutting deformation and inferior rutting resistance. AC-16 steel slag asphalt mixture has superior resistance to permanent deformation than AC-20 asphalt mixture. Rutting factors show different degrees of impact in a decreasing order of temperature, water damage, and stress levels. The findings have significant implications for providing a theoretical basis for reusing steel slag in pavement construction and facilitating engineering application of steel slag asphalt mixture.