We propose a polarimetric technique for a stellar interferometer that uses two birefringent retarders inserted into each arm of the interferometer. The retarders shift the interference fringes of the two orthogonal polarizations to opposite directions along an optical path difference (OPD), which enables us to measure the polarized fringe visibilities separately. Laboratory demonstrations were carried out using quartz retarders with a thickness of 1.124 mm, and the separation of the interferometric fringes of +/-10 microm along the OPD was observed. We discuss a calibration method for the acquired fringes to obtain reliable estimations of polarized fringe visibilities. We also discuss feasibility of the proposed technique for the Mitaka optical InfraRed Array (MIRA-I.2).