The evolution of microstructure and residual stress during the tempering of 700 L low-carbon micro-alloyed steel was studied using a crack compliance method for measuring residual stress. Additionally, a non-isothermal tempering dilatation test, Vickers micro-hardness test, and transmission electron microscopy were used. The evolution of residual stress during tempering consists of two stages. The first stage coincided with cementite precipitation. Under the initial residual stress, the transformation plasticity due to cementite precipitation leads to partial relaxation of the micro-stress evoked by the austenite-to-ferrite transformation during quenching. It also caused the material surface and the core to exhibit different residual stress evolution trends. After tempering at 300 ∘ C for 30 min, the residual stress was reduced from 487 MPa to 200 MPa; however, the elastic strain energy remained unchanged. The second stage coincided with alloy carbide precipitation and Mn partitioning, but the precipitation of the alloy carbide only reduced the elastic strain energy by 8.7%. Thus, the change in activation energy was the main reason for the relaxation of residual stress at this stage. After tempering at 600 ∘ C for 30 min, the residual stress was reduced to 174 MPa, the elastic strain energy was reduced by 72.72%, and the residual stress was controlled.