Prediction, Retrodiction, and the Amount of Information Stored in the Present

Abstract

We introduce an ambidextrous view of stochastic dynamical systems, comparing their forward-time and reverse-time representations and then integrating them into a single time-symmetric representation. The perspective is useful theoretically, computationally, and conceptually. Mathematically, we prove that the excess entropy-a familiar measure of organization in complex systems-is the mutual information not only between the past and future, but also between the predictive and retrodictive causal states. Practically, we exploit the connection between prediction and retrodiction to directly calculate the excess entropy. Conceptually, these lead one to discover new system measures for stochastic dynamical systems: crypticity (information accessibility) and causal irreversibility. Ultimately, we introduce a time-symmetric representation that unifies all of these quantities, compressing the two directional representations into one. The resulting compression offers a new conception of the amount of information stored in the present.

Informational and Causal Architecture of Continuous-time Renewal Processes

“…Ref. [12].) The forwardtime and reverse-time statistical complexities are the entropies of S + and S − , i.e., the memories required to losslessly predict or retrodict, respectively.…”

“…[12,13] shows that E = I[S + ; S − ] by applying the causal shielding relations in Eqs. (1) and (2).…”

“…Together, a process' causal irreversibility [12,13] is defined as the difference between the forward and reversetime statistical complexities:…”

“…With this restriction on labeled transition matrices, the time-reversed -machine of the process has the same form as the -machine of the forward-time process, but with a different G. The latter is related to the forward-time G via manipulations described in Ref. [12]. As such, we can write down p(σ + , σ − ):…”

“…Pr(σ + = (τ, g, x)|σ − = (g , x , τ )) = Pr(τ |g, x, τ )δ x,x Pr(g|g , x , x, τ ) = φ g,x (τ + τ ) Pr(g|g , x)δ x,x , where we obtain p(g|g , x) from standard methods [12,26] applied to (only) the dynamic on G. Note that p(g|g , x , x, τ ) reduces to p(g|g , x ) as g and x uniquely specify the distribution from which τ is drawn and since x = x . We leave the the final steps to E as an exercise.…”