We demonstrate that the linearity of silicon ring modulators can be improved significantly by manipulating the dynamics of photons in the cavity. A theoretically model is established to analyze the dependence of the modulation linearity on lifetime and wavelength of photons. By reducing Q factors of silicon ring modulators from 11000 to 5880 and tuning the operation wavelength, measured Spurious-Free Dynamic Ranges for the third-order intermodulation distortion are improved from 98.5 dB•Hz 2/3 to 104.3 dB•Hz 2/3 and from 90.6 dB•Hz 2/3 to 94.7 dB•Hz 2/3 at 1 GHz and 10 GHz, respectively.