The cracking resistance of asphalt mixture is a non-negligible issue. However, the cracking resistance evolution law, motivated by two factors (thermos-oxidative aging degree and test temperature), is not yet well understood. The aim of this investigation is to gain more insight into the effect of thermos-oxidative aging and test temperature on the cracking resistance of asphalt mixture. Asphalt mixture (AC-13) and stone mastic asphalt mixture (SMA-13) were selected and exposed to different thermo-oxidative aging degrees (unaging (UA); short-term thermo-oxidative aging (STOA); long-term thermo-oxidative aging for 2/5/8 days (LTOA2d/LTOA5d/LTOA8d)). A direct tension test at different test temperatures (10 °C, 20 °C, 30 °C, 40 °C) was adopted to obtain their stress–strain curves and evaluation indexes (tensile strength, ultimate strain, pre-peak strain energy density, and post-peak strain energy density). The comprehensive index-cracking resistance index (CRI) was established by the entropy weight method combined with the technique to order preference by similarity to ideal solution (TOPSIS) method and the corresponding aging coefficient was determined. The results showed that STOA can increase the aging coefficient of asphalt mixture, thereby boosting the cracking resistance. Additionally, the effect can be weakened by elevations in the test temperature. Meanwhile, LTOA can decrease the aging coefficient and thereby weaken the cracking resistance. This effect becomes more prominent with elevations in the test temperature. SMA-13 possesses a superior cracking resistance to AC-13, with a gap in CRI value of 3–69%, regardless of the aging degree and test temperature. A good relationship exists between the aging coefficient and the two factors (aging degree and test temperature).