Steel slag’s physical and mechanical properties are close to natural gravel, which can replace natural aggregate with paving asphalt mixture pavement. Based on macro- and microscales, it is necessary to study the low-temperature cracking characteristics of steel slag asphalt mixture (SAM). This paper aims to study the low-temperature cracking characteristics and influence mechanism of steel slag asphalt mixture by macro- and microscale tests. This article analyzes the strain field distribution and cracks propagation law of SAM at low temperatures by the three-point bending test combined with digital image correlation (DIC). The microscopic pore characteristics of SAM were studied by nuclear magnetic resonance (NMR). Based on fracture mechanics and weight function theory, the variation law of the stress intensity factor at the crack tip of the asphalt mixture with crack propagation is derived. The correlation between microscopic pores and low-temperature cracking characteristics of asphalt mixture was analyzed based on the grey correlation theory. The results show that SAM has more small and less large pores, and its total porosity is lesser than that of the basalt asphalt mixture (BAM). The initial elastic stage of SAM resisting bending deformation is longer than that of BAM, the bending strain energy density (dW/dV) is larger, and the crack penetration is slower. The horizontal strain energy density (DE) of SAM is greater than BAM, and the crack tip stress is greater. The porosity correlates well with the low-temperature crack resistance of the asphalt mixture.