Limits on inferring the past

Abstract

Here we define and study the properties of retrodictive inference. We derive equations relating retrodiction entropy and thermodynamic entropy, and as a special case, show that under equilibrium conditions, the two are identical. We demonstrate relations involving the Kullback-Leibler divergence and retrodiction probability, and bound the time rate of change of retrodiction entropy. As a specific case, we invert various Langevin processes, inferring the initial condition of N particles given their final positions at some later time. We evaluate the retrodiction entropy for Langevin dynamics exactly for special cases, and find that one's ability to infer the initial state of a system can exhibit two possible qualitative behaviors depending on the potential energy landscape, either decreasing indefinitely, or asymptotically approaching a fixed value. We also study how well we can retrodict points that evolve based on the logistic map. We find singular changes in the retrodictivity near bifurcations. Counterintuitively, the transition to chaos is accompanied by maximal retrodictability.

Enhancing the predictability and retrodictability of stochastic processes

“…Interestingly, the predictability and retrodictability of a system are tightly connected: Since S T and S R are related by the Bayes' theorem, R ω (α) = T α (ω)P (0) (α)/ α T α (ω)P (0) (α ), it follows that S T and S R are also related [31],…”

“…Forward in time, the entropy associated with the probability distribution of the system state will increase monotonically, as per H−theorems [33][34][35]. A similar trend also holds backwards in time [31]. Thus, our task is to determine how transition rates should be perturbed infinitesimally as to minimize the generation of inferential entropy in either temporal direction.…”

“…A number of authors addressed the problem of predicting the future and retrodicting the past of a stochastic process [28][29][30][31]. In this study, we are concerned not with finding specific strategies or algorithms to optimally predict or retrodict stochastic systems, but rather finding ways to optimally modify stochastic systems so that they become more predictabile or retrodictable.…”