Abstract-Parsimonious parametric models for nonstationary random processes are useful in many applications. Here, we consider a nonstationary extension of the classical autoregressive moving-average (ARMA) model that we term the time-frequency autoregressive moving-average (TFARMA) model. This model uses frequency shifts in addition to time shifts (delays) for modeling nonstationary process dynamics. The TFARMA model and its special cases, the TFAR and TFMA models, are shown to be specific types of time-varying ARMA (AR, MA) models. They are attractive because of their parsimony for underspread processes, that is, nonstationary processes with a limited time-frequency correlation structure. We develop computationally efficient order-recursive estimators for the TFARMA, TFAR, and TFMA model parameters which are based on linear time-frequency Yule-Walker equations or on a new time-frequency cepstrum. Simulation results demonstrate that the proposed parameter estimators outperform existing estimators for time-varying ARMA (AR, MA) models with respect to accuracy and/or numerical efficiency. An application to the time-varying spectral analysis of a natural signal is also discussed.