To eliminate the serious detrimental effects caused by frequency drifts of acousto-optic frequency shifters (AOFSs) in conventional all-fiber-optic heterodyne Doppler measurement systems, a differential allfiber-optic heterodyne configuration is proposed. By delicate arrangement of the fiber-optic path and using only one AOFS, zero drift, which commonly occurs in conventional systems, is eliminated. Based on the proposed configuration, a differential all-fiber-optic heterodyne Doppler measurement system has been built. Using a piezoelectric ceramic oscillator as the moving object, it has been verified that the proposed system is able to eliminate zero drift, and the displacement and velocity measurement accuracies reach 0.775 μm and 0.009 mm∕s, respectively. It has been shown that the differential all-fiber-optic heterodyne Doppler measurement system can achieve good performance in both displacement and velocity measurements, even in a harsh environment with drastic temperature variation.