Kalman-like state tracking and control in POMDPS with applications to body sensing networks

Abstract

In this paper, the problem of state tracking with controlled observations is considered for a system modeled by a discrete-time, finitestate Markov chain. The system state is 'hidden' and observed via conditionally Gaussian measurements that are shaped by the underlying state and an exogenous control input. Following an innovations approach, a Kalman-like filter is derived to estimate the Markov chain system state. To optimize the control strategy, the associated mean-squared error is used as an optimization criterion for a partially observable Markov Decision Process (POMDP). The optimal solution is determined via stochastic dynamic programming. Numerical results are presented for the application of physical activity detection in heterogeneous, wireless body area networks.Index Terms-Markov models, approximate MMSE, POMDP, stochastic dynamic programming, innovations approach

A unified framework for energy efficient physical activity tracking

“…, p n k|k ] T denote the pmf of x k conditioned on F k with p i k|k = P (x k = e i |F k ), ∀e i ∈ X. In [3], we showed that this pmf coincides with the MMSE estimate of x k conditioned on F k and derived the approximate, nonlinear MMSE estimator given in Theorem 1.…”

“…Specifically, we introduce our POMDP [14] formulation and we review our previously proposed Kalmanlike estimator [3], which constitutes a key component of the proposed framework.…”

“…In [3], we developed an approximate nonlinear MMSE estimator for the Markov chain system state. This estimator is reviewed in this section.…”

“…Due to the way the proposed estimator was designed, it can give rise to invalid pmfs, and to avoid this situation, we apply an appropriate transformation [3]. We underscore that the pmf p k|k (also known as belief state [14]) constitutes an estimate of the individual's unknown activity state and is driven by the control input selection.…”

“…Thus, we devise sensor selection strategies for the latter to optimize the trade-off between tracking performance and energy consumption. We adopt our recently proposed approximate minimum mean-squared error (MMSE) estimator [3] for physical activity tracking. For This research has been funded in part by the following grants and organizations: ONR N00014-09-1-0700, NSF CNS-0832186, CCF-0917343, CCF-1117896, CNS-1213128, AFOSR FA9550-12-1-0215, DOT CA-26-7084-00, the National Center on Minority Health and Health Disparities (NCMHD) (supplement to P60 MD002254), Nokia and Qualcomm.…”