Estimation in slow mixing, long memory channels

Abstract

We consider estimation of binary channels with memory where the transition probabilities (channel parameters) from the input to output are determined by prior outputs (state of the channel). While the channel is unknown, we observe the joint input/output process of the channel-we have n i.i.d. input bits and their corresponding outputs. Motivated by applications related to the backplane channel, we want to estimate the channel parameters as well as the stationary probabilities for each state.Two distinct problems complicate estimation in this setting: (i) long memory, and (ii) slow mixing which could happen even with only one bit of memory. In this setting, any consistent estimator can only converge pointwise over the model class. Namely, given any estimator and any sample size n, the underlying model could be such that the estimator performs poorly on a sample of size n with high probability. But can we look at a length-n sample and identify if an estimate is likely to be accurate?Since the memory is unknown a-priori, a natural approach, known to be consistent, is to estimate a potentially coarser model with memory kn = αn log n, where αn is a function that grows O(1). Note however that (i) the coarser model is estimated using only samples from the true model; and (ii) we want the best possible answers with a length-n sample, rather than just consistency. Combining results on universal compression and Aldous' coupling arguments, we obtain sufficient conditions (even for slow mixing models) to identify when naive (i) estimates of the channel parameters and (ii) estimates related to the stationary probabilities of the channel states are accurate, and bound their deviations from true values.

Agnostic insurability of model classes

“…We want to deal with broader classes of models, and the flexibility about the start point for prediction allows us to consider significantly richer model classes. For other kinds of such "useful" pointwise estimation, particularly in relation to Markov processes, see Asadi et al (2013).…”

On the Performance of Block Codes Over Finite-State Channels in the Rare-Transition Regime

“…It is worth pointing out that the rare-transition regime, also known as the slow-mixing regime, has been studied in other contexts such as channel estimation, asymptotic filtering, and entropy rate analysis of hidden Markov processes [8], [9], [10], [11]. In this article, we examine probabilities of decoding failure, their distributions and temporal properties within the context of rare-transition regime.…”