In this paper, we address the problem of designing power efficient quantizers for state estimation of hidden Markov models using multiple sensors communicating to a fusion centre via error-prone randomly time-varying flat fading channels modelled by finite state Markov chains. Our objective is to minimize a tradeoff between the long term average of mean square estimation error and expected total power consumption. We formulate the problem as a stochastic control problem by using Markov decision processes. Under some mild assumption on the measurement noise at the sensors, the discretized action space (quantization thresholds and transmission power levels) version of the optimization problem forms a unichain Markov decision process for stationary policies. The solution to the discretized problem provides optimal quantization thresholds and power levels to be communicated back to the sensors via a feedback channel. Moreover, in order to improve the performance of the quantization system, we employ a gradientfree stochastic optimization technique to determine the optimal set of quantization thresholds from which optimal quantization levels are determined. The performance results for estimation error/total transmission power tradeoff are studied under various channel conditions and sensor measurement qualities.

Isr develops, applies and teaches advanced methodologies of design and analysis to solve complex, hierarchical, heterogeneous and dynamic problems of engineering technology and systems for industry and government.Isr is a permanent institute of the university of maryland, within the a. James clark school of engineering. It is a graduated national science foundation engineering research center. Abstract-This paper studies almost sure convergence of a dynamic average consensus algorithm which allows distributed computation of the product of n time-varying conditional probability density functions. These conditional probability density functions (often called as "belief functions") correspond to the conditional probability of observations given the state of an underlying Markov chain, which is observed by n different nodes within a sensor network. The network topology is modeled as an undirected graph. The average consensus algorithm is used to obtain a distributed state estimation scheme for a hidden Markov model (HMM), where each sensor node computes a conditional probability estimate of the state of the Markov chain based on its own observations and the messages received from its immediate neighbors. We use the ordinary differential equation (ODE) technique to analyze the convergence of a stochastic approximation type algorithm for achieving average consensus with a constant step size. This allows each node to track the time varying average of the logarithm of conditional observation probabilities available at the individual nodes in the network. It is shown that, for a connected graph, under mild assumptions on the first and second moments of the observation probability densities and a geometric ergodicity condition on an extended Markov chain, the consensus filter state of each individual sensor converges P-a.s. to the true average of the logarithm of the conditional observation probability density functions of all the sensors. Convergence is proved by using a perturbed stochastic Lyapunov function technique. Numerical results suggest that the distributed Markov chain state estimates obtained at the individual sensor nodes based on this consensus algorithm track the centralized state estimate (computed on the basis of having access to observations of all the nodes) quite well, while formal results on convergence of the distributed HMM filter to the centralized one are currently under investigation.

This paper is concerned with dynamic quantizer design for state estimation of hidden Markov models (HMM) using multiple sensors under a sum power constraint at the sensor transmitters. The sensor nodes communicate with a fusion center over temporally correlated flat fading channels modelled by finite state Markov chains. Motivated by energy limitations in sensor nodes, we develop optimal quantizers by minimizing the long term average of the mean square estimation error with a constraint on the long term average of total transmission power across the sensors. Instead of introducing a cost function as a weighted sum of our two objectives, we propose a constrained Markov decision formulation as an average cost problem and employ a linear programming technique to obtain the optimal policy for the constrained problem. Our experimental results assert that the constrained approach is quite efficient in terms of computational complexity and memory requirements for our average cost problem and leads to the same optimal deterministic policies and optimal cost as the unconstrained approach under an irreducibility assumption on the underlying Markov chain and some mild regularity assumptions on the sensor measurement noise processes. We illustrate via numerical studies the performance results for the dynamic quantization scheme. We also study the effect of varying degrees of channel and measurement noise on the performance of the proposed scheme.

This paper investigates an optimal quantizer design problem for multisensor estimation of a hidden Markov model (HMMs) whose description depends on unknown parameters. The sensor measurements are simply binary quantized and transmitted to a remote fusion center over noisy flat fading wireless channels under an average sum transmit power constraint. The objective is to determine a set of optimal quantization thresholds and sensor transmit powers, called an optimal policy, which minimizes the long run average of a weighted combination of the expected state estimation error and sum transmit power. We analyze the problem by formulating an adaptive Markov decision process (MDP) problem. In this framework, adaptive optimal control policies are obtained using a nonstationary value iteration (NVI) scheme and are termed as NVI-adaptive policies. These NVI-adaptive policies are adapted to the HMM parameter estimates obtained via a strongly consistent maximum likelihood estimator. In particular, HMM parameter estimation is performed by a recursive expectation-maximization (EM) algorithm which computes estimates of the HMM parameters by maximizing a relative entropy information measure using the received quantized observations and the trajectory of the MDP. Under some regularity assumptions on the observation probability distributions and a geometric ergodicity condition on an extended Markov chain, the maximum-likelihood estimator is shown to be strongly consistent. It is shown that the NVI-adaptive policy based on this sequence of strongly consistent HMM parameter estimates is (asymptotically, under appropriate assumptions) average-optimal. Essentially, it minimizes the long run average cost of the weighted combination of the expected state estimation error and sum transmit power across the sensors for the HMM with true parameters in a time-asymptotic sense. The advantage of this scheme is that the policies are obtained recursively without the need to solve the Bellman equation at each time step, which can be computationally prohibitive. As is usual with value iteration schemes, practical implementation of the NVI-adaptive policy requires discretization of the state and action space, which results in some loss of optimality. Nevertheless, numerical results illustrate the asymptotic convergence properties of the parameter estimates and the asymptotically close to optimal performance of the adaptive MDP algorithm compared to the performance of an MDP based dynamic quantization and power allocation algorithm designed with perfect knowledge of the true parameters.

This paper addresses an estimation problem for hidden Markov models (HMMs) with unknown parameters, where the underlying Markov chain is observed by multiple sensors. The sensors communicate their binary-quantized measurements to a remote fusion centre over noisy fading wireless channels under an average sum transmit power constraint. The fusion centre minimizes the expected state estimation error based on received (possibly erroneous) quantized measurements to determine the optimal quantizer thresholds and transmit powers for the sensors, called the optimal policy, while obtaining strongly consistent parameter estimates using a recursive maximum likelihood (ML) estimation algorithm. The problem is formulated as an adaptive Markov decision process (MDP) problem. To determine an optimal policy, a stationary policy is adapted to the estimated values of the true parameters. The adaptive policy based on the maximum likelihood estimator is shown to be average optimal. A nonstationary value iteration scheme is employed to obtain adaptive optimal policies which has the advantage that the policies are obtained recursively without the need to solve the Bellman optimality equation at each time step. We provide some numerical examples to illustrate the analytical results.