The snapshot imaging polarimeters (SIPs) using spatial modulation have gained increasing popularity due to their capability of obtaining all four Stokes parameters in a single measurement. However, the existing reference beam calibration techniques cannot extract the modulation phase factors of the spatially modulated system. In this paper, a calibration technique based on a phase-shift interference (PSI) theory is proposed to address this issue. The proposed technique can accurately extract and demodulate the modulation phase factors through measuring the reference object at different polarization analyzer orientations and performing a PSI algorithm. Using the snapshot imaging polarimeter with modified Savart polariscopes as an example, the basic principle of the proposed technique is analyzed in detail. Subsequently, the feasibility of this calibration technique was demonstrated by a numerical simulation and a laboratory experiment. This work provides a different perspective for the calibration of a spatially modulated snapshot imaging polarimeter.