In this paper, we propose a robust time-frequency decomposition (RTFD) model to restore audio signals degraded by sparse impulse noise mixed with small dense Gaussian noise. This kind of noise is very common especially in old-time recordings. The proposed RTFD model is based on the observation that these degraded audio signals mainly contain four parts, i.e., the quasi-periodic and voiced part, the aperiodic and transient part, the arbitrarily large impulse noise and the small dense Gaussian noise. Sparsity and local correlations of corresponding parts are exploited to solve the RTFD model. We also heuristically develop a discriminative orthogonal matching pursuit (DOMP) algorithm to more precisely estimate sparse representing vectors. Specifically, the DOMP algorithm divides the whole atom set into two subsets, i.e., the active subset and the passive subset. Atoms in two subsets are treated discriminatively since sparsity regularization terms are not equally weighted. Based on RTFD and DOMP, we have developed two algorithms, i.e., the fidelity-oriented algorithm and the articulation-oriented algorithm. The proposed algorithms achieve considerable performance on both synthetic and real noisy signals. Results show that the articulation-oriented algorithm using DOMP obviously outperforms other algorithms in heavier impulse noise situations.Index Terms-Articulation-oriented, degraded, discriminative orthogonal matching pursuit (DOMP), Gaussian-impulse noise, restoration, robust time-frequency decomposition (RTFD).