Low-frequency vibration is a harmful factor that affects the accuracy of micro/nano-measuring machines. Low-frequency vibration cannot be completely eliminated by passive control methods, such as the use of air-floating platforms. Therefore, low-frequency vibrations must be measured before being actively suppressed. In this study, the design of a low-cost high-sensitivity optical accelerometer is proposed. This optical accelerometer mainly comprises three components: a seismic mass, a leaf spring, and a sensing component based on a four-quadrant photodetector (QPD). When a vibration is detected, the seismic mass moves up and down due to the effect of inertia, and the leaf spring exhibits a corresponding elastic deformation, which is amplified by using an optical lever and measured by the QPD. Then, the acceleration can be calculated. The resonant frequencies and elastic coefficients of various seismic structures are simulated to attain the optimal detection of low-frequency, low-amplitude vibration. The accelerometer is calibrated using a homemade vibration calibration system, and the calibration experimental results demonstrate that the sensitivity of the optical accelerometer is 1.74 V (m·s−2)−1, the measurement range of the accelerometer is 0.003–7.29 m·s−2, and the operating frequencies range of 0.4–12 Hz. The standard deviation from ten measurements is under 7.9 × 10−4 m·s−2. The efficacy of the optical accelerometer in measuring low-frequency, low-amplitude dynamic responses is verified.