A non-parametric hidden Markov model for climate state identification

Abstract

Hidden Markov models (HMMs) can allow for varying wet and dry cycles in the climate without the need to simulate supplementary climate variables. The fitting of a parametric HMM relies upon assumptions for the state conditional distributions. It is shown that inappropriate assumptions about state conditional distributions can lead to biased estimates of state transition probabilities. An alternative non-parametric model with a hidden state structure that overcomes this problem is described. It is shown that a two-state non-parametric model produces accurate estimates of both transition probabilities and the state conditional distributions. The non-parametric model can be used directly or as a technique for identifying appropriate state conditional distributions to apply when fitting a parametric HMM. The non-parametric model is fitted to data from ten rainfall stations and four streamflow gauging stations at varying distances inland from the Pacific coast of Australia. Evidence for hydrological persistence, though not mathematical persistence, was identified in both rainfall and streamflow records, with the latter showing hidden states with longer sojourn times. Persistence appears to increase with distance from the coast.

Parameter estimation and model identification for stochastic models of annual hydrological data: Is the observed record long enough?

“…If the transition probabilities sum to one, p WD + p DW = 1 then there is no persistence between states and the HMM degenerates to a mixture model (Lambert et al, 2003). Therefore, to determine whether the HMM has been identified it is necessary to evaluate the probability of the sum of the transition probabilities exceeding one, P(p WD + p DW > 1).…”

“…This provides an explicit mechanism to simulate hydrological time series with long-term wet and periods which are often evident in long-term hydrological data. Thyer and Kuczera (2000) and Lambert et al (2003) explain in detail how a hydrological time series is simulated using a single-site HMM. Only a brief description is given here.…”

“…The parameter uncertainty will be summarized using the range of the 90% probability limits (PL) of the posterior, based on the approach of Lambert et al (2003). The 90% PL range is defined as the difference between the upper 95% and lower 5% PL of the posterior.…”

“…Hereafter in this paper the two-state HMM will simply be referred to as HMM. Lambert et al (2003) developed a non-parametric HMM (NPHMM) for the modelling of monthly time series. Their motivation for developing a non-parametric approach was to identify parametric state distributions suitable for monthly data.…”

A general Bayesian framework for calibrating and evaluating stochastic models of annual multi-site hydrological data

“…If the transition probabilities sum to one, p WD + p DW = 1 then there is no persistence between states and the HMM degenerates to a mixture model (Lambert et al, 2003). The HMM can also produce the behaviour observed in shifting mean and change-point models (low transition probabilities can produce a single change-point within a series).…”