We present a design for a quantum key distribution (QKD) system in a Sagnac loop configuration, employing a novel phase modulation scheme based on frequency shift, and demonstrate stable BB84 QKD operation with high interference visibility and low quantum bit error rate (QBER). The phase modulation is achieved by sending two light pulses with a fixed time delay (or a fixed optical path delay) through a frequency shift element and by modulating the amount of frequency shift. The relative phase between two light pulses upon leaving the frequency-shift element is determined by both the time delay (or the optical path delay) and the frequency shift, and can therefore be controlled by varying the amount of frequency shift. To demonstrate its operation, we used an acousto-optic modulator (AOM) as the frequency-shift element, and vary the driving frequency of the AOM to encode phase information. The interference visibility for a 40km and a 10km fiber loop is 96% and 99%, respectively, at single photon level. We ran BB84 protocol in a 40-km Sagnac loop setup continuously for one hour and the measured QBER remained within the 2%∼5% range. A further advantage of our scheme is that both phase and amplitude modulation can be achieved simultaneously by frequency and amplitude modulation of the AOM's driving signal, allowing our QKD system the capability of implementing other protocols, such as the decoy-state QKD and the continuous-variable QKD. We also briefly discuss a new type of Eavesdropping strategy ("phaseremapping" attack) in bidirectional QKD system.