This paper describes the design and performance of a low-noise and high-speed optical sensor that provides two output signals in quadrature from the simultaneous detection of four phase-shifted interferograms. The sensor employs four high-speed photodiodes and high-speed, low-noise transimpedance amplifiers. The optical and electronic design was optimized for high-speed displacement measurement interferometry, over a broad range of operating frequencies. Compared to other experimental schemes, the sensor is simpler and of lower cost. The performance of the sensor is demonstrated by characterizing a piezoelectric transducer for ultrasonic applications. We measured displacements between 38 pm and 32 nm with 6% relative uncertainty, in the frequency range from 1 to 2 MHz.