Gear fault related information is distributed over a broad frequency band, indicating a complex modulation mechanism. It is difficult to detect early-stage gear faults accurately by detecting fault frequencies in a limited frequency band. This paper proposes a novel method for achieving fault frequency detection more effectively. A short-frequency Fourier transform with a series of frequency-window functions is initially used to obtain the overall frequency information of a vibration signal. Subsequently, based on sparse decomposition and orthogonal matching pursuit, harmonic atoms are applied to refine modulation components from multiscale pseudo mono-components. A multiscale-sparse frequencyfrequency distribution is eventually applied to augment existing fault-related harmonic components. In addition, a synthesized sparse spectrum is acquired by determining the frequency-frequency ridge from the multiscale sparse frequency-frequency distribution. Compared with empirical-mode-decomposition and fast-kurtogram analyses, the effectiveness and superiority of the proposed method for gear fault detection have been verified via experiments.