Statistical signatures of structural organization: The case of long memory in renewal processes

Abstract

Identifying and quantifying memory are often critical steps in developing a mechanistic understanding of stochastic processes. These are particularly challenging and necessary when exploring processes that exhibit long-range correlations. The most common signatures employed rely on second-order temporal statistics and lead, for example, to identifying long memory in processes with power-law autocorrelation function and Hurst exponent greater than 1/2. However, most stochastic processes hide their memory in higher-order temporal correlations. Information measuresspecifically, divergences in the mutual information between a process' past and future (excess entropy) and minimal predictive memory stored in a process' causal states (statistical complexity)-provide a different way to identify long memory in processes with higher-order temporal correlations. However, there are no ergodic stationary processes with infinite excess entropy for which information measures have been compared to autocorrelation functions and Hurst exponents. Here, we show that fractal renewal processes-those with interevent distribution tails ∝ t −α -exhibit long memory via a phase transition at α = 1. Excess entropy diverges only there and statistical complexity diverges there and for all α < 1. When these processes do have power-law autocorrelation function and Hurst exponent greater than 1/2, they do not have divergent excess entropy. This analysis breaks the intuitive association between these different quantifications of memory. We hope that the methods used here, based on causal states, provide some guide as to how to construct and analyze other long memory processes.

Informational and Causal Architecture of Continuous-time Renewal Processes

“…There, the interevent interval is so structured that the resultant process can have power-law correlations [17]. Then, knowing the time since last event can provide quite a bit of predictive power [8].…”

“…(Elsewhere, we outline the wide interest and applicability of renewal processes in physics and the quantitative sciences generally [7][8][9].) The difficulties are both technical and conceptual.…”

Thermodynamics of random number generation

“…Second, the sources of randomness and target distributions we considered were rather limited when compared to the very wide range of stochastic processes used in contemporary science. For example, what about the thermodynamics of generating ated with infinite memory processes [89]. What are associated thermodynamic cost bounds?…”

Information trimming: Sufficient statistics, mutual information, and predictability from effective channel states

“…[25,26], for example. While closed-form expressions for excess entropy of finite-state processes have existed for several years [2,13], it is only recently that it has been analyzed for truly complex (infinitestate) processes [25,26]. In this work, identifying and then framing calculations around the causal states led to substantial progress.…”