A common approach for modeling categorical time series is Hidden-Markov models (HMMs), where the actual observations are assumed to depend on hidden states in their behavior and transitions. Such categorical HMMs are even applicable to nominal data but suffer from a large number of model parameters. In the ordinal case, however, the natural order among the categorical outcomes offers the potential to reduce the number of parameters while improving their interpretability at the same time. The class of ordinal HMMs proposed in this article link a latent-variable approach with categorical HMMs. They are characterized by parametric parsimony and allow the easy calculation of relevant stochastic properties, such as marginal and bivariate probabilities. These points are illustrated by numerical examples and simulation experiments, where the performance of maximum likelihood estimation is analyzed in finite samples. The developed methodology is applied to real-world data from a health application.