An application of the information theory to filtering problems

Tsutsumi, Yutaka; Omatu, S.; Soeda, Takashi

doi:10.1016/0020-0255(76)90034-7

Cited by 21 publications

(21 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, a variety of information theoretic measures of optimality such as the mutual information between the observation and the estimation error are considered in [30], [14] and [8]. In particular, these articles show that many of these measures are optimised by the Kalman filter in the linear Gaussian filtering problem of this article.…”

Section: Introductionmentioning

confidence: 99%

Information and Entropy Flow in the Kalman?Bucy Filter

Mitter

Newton

2005

J Stat Phys

View full text Add to dashboard Cite

We investigate the information theoretic properties of KalmanBucy filters in continuous time, developing notions of information supply, storage and dissipation. Introducing a concept of energy, we develop a physical analogy in which the unobserved signal describes a statistical mechanical system interacting with a heat bath. The abstract 'universe' comprising the signal and the heat bath obeys a non-increase law of entropy; however, with the introduction of partial observations, this law can be violated. The Kalman-Bucy filter behaves like a Maxwellian demon in this analogy, returning signal energy to the heat bath without causing entropy increase. This is made possible by the steady supply of new information.In a second analogy the signal and filter interact, setting up a stationary non-equilibrium state, in which energy flows between the heat bath, the signal and the filter without causing any overall entropy increase. We introduce a rate of interactive entropy flow that isolates the statistical mechanics of this flow from marginal effects. Both analogies provide quantitative examples of Landauer's Principle.

show abstract

Section: Introductionmentioning

confidence: 99%

Information and Entropy Flow in the Kalman?Bucy Filter

Mitter

Newton

2005

J Stat Phys

View full text Add to dashboard Cite

show abstract

“…The above equation can also be expressed in entropy terms as follows (Tomita et al 1976): where, H(Zt) is the entropy of variable Zt;…”

Section: Formulation Of Kalman Filter Estimation Modelmentioning

confidence: 99%

“…In which variable Y is the m-dimensional vector. Tomita et al (1976) used this error entropy concept and estimated the matrices At and St+1 as follows:…”

Section: Formulation Of Kalman Filter Estimation Modelmentioning

confidence: 99%

Meteorological Data Simulation Using Kalman Filter Estimation Model

Husain¹,

Ukayli²,

Khan³

1983

Journal of the Meteorological Society of Japan

View full text Add to dashboard Cite

show abstract

“…Similar quantification is performed for non-Gaussian signal [9,10] and fractional Gaussian channel [11]. On the other hand, [12] showed that the optimal filter for a linear system that maximizes the mutual information between the observation history for [0, t] and the state value at t is Kalman-Bucy filter; [13] related this mutual information to the Fisher information matrix. Recently, Mitter and Newton [14] presented an expression for the mutual information between the signal path during [s, t], s < t and the observation history, with statistical mechanical interpretation of this expression.…”

Section: Introductionmentioning

confidence: 99%

“…Regarding the quantification of the mutual information, this work first presents the filter form, which is a simple extension of the previous work [12]- [14], by treating the forecast problem as a filtering problem with longer time window. However, it then shows that this form might not be suitable for motion planning for a long-term forecast in three senses: sensitivity to the model accuracy, computational cost, and the lack of on-the-fly knowledge of the accumulated information.…”

Section: Introductionmentioning

confidence: 99%

Continuous motion planning for information forecast

Choi

How

2008

2008 47th IEEE Conference on Decision and Control

View full text Add to dashboard Cite

Abstract-This paper addresses planning of continuous paths for mobile sensors to improve long-term forecast performance. With the information gain defined by the mutual information between the continuous measurement path and the future verification variables, two expressions for computing the information gain for a linear time-varying system are derived: the filter form and the smoother form. The smoother form, inspired by the conditional independence structure, is shown to be preferable, since it does not require integration of differential equations for long time intervals, it simplifies the process of calculating the accumulated information on the fly, and its time derivative extracts out the pure impact of sensing regardless of the process noise. Utilizing the spatial interpolation technique to relate the sensor movement to the evolution of the observation matrix, the optimal path planning formulation and the real-time steering law are presented. A numerical example of a simplified weather forecast validates the proposed methodology.

show abstract

An application of the information theory to filtering problems

Cited by 21 publications

References 7 publications

Information and Entropy Flow in the Kalman?Bucy Filter

Information and Entropy Flow in the Kalman?Bucy Filter

Meteorological Data Simulation Using Kalman Filter Estimation Model

Continuous motion planning for information forecast

Contact Info

Product

Resources

About