Signatures of infinity: Nonergodicity and resource scaling in prediction, complexity, and learning

Crutchfield, James P.; Marzen, Sarah

doi:10.1103/physreve.91.050106

Cited by 13 publications

(10 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To close, let's return to our initial discussion of statistical signatures of structural organization. We drew a comparison of divergent memory in ergodic processes to that we previously identified in the so-called Bandit nonergodic processes [26]. The mechanism underlying the latter was rather straightforward: from trial to trial the process remembers the operant ergodic component subprocess and so uses an infinite memory and exhibits an excess entropy that diverges as log .…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Statistical signatures of structural organization: The case of long memory in renewal processes

Marzen¹,

Crutchfield²

2016

Physics Letters A

Self Cite

View full text Add to dashboard Cite

Identifying and quantifying memory are often critical steps in developing a mechanistic understanding of stochastic processes. These are particularly challenging and necessary when exploring processes that exhibit long-range correlations. The most common signatures employed rely on second-order temporal statistics and lead, for example, to identifying long memory in processes with power-law autocorrelation function and Hurst exponent greater than 1/2. However, most stochastic processes hide their memory in higher-order temporal correlations. Information measuresspecifically, divergences in the mutual information between a process' past and future (excess entropy) and minimal predictive memory stored in a process' causal states (statistical complexity)-provide a different way to identify long memory in processes with higher-order temporal correlations. However, there are no ergodic stationary processes with infinite excess entropy for which information measures have been compared to autocorrelation functions and Hurst exponents. Here, we show that fractal renewal processes-those with interevent distribution tails ∝ t −α -exhibit long memory via a phase transition at α = 1. Excess entropy diverges only there and statistical complexity diverges there and for all α < 1. When these processes do have power-law autocorrelation function and Hurst exponent greater than 1/2, they do not have divergent excess entropy. This analysis breaks the intuitive association between these different quantifications of memory. We hope that the methods used here, based on causal states, provide some guide as to how to construct and analyze other long memory processes.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Since strongly mixing processes have short memory and nonergodic processes could be said to have infinite memory [26], Ref. [20] proposed that one or another type of nonmixing property is a good candidate for long memory in ergodic stationary processes.…”

Section: Introductionmentioning

confidence: 99%

Statistical signatures of structural organization: The case of long memory in renewal processes

Marzen¹,

Crutchfield²

2016

Physics Letters A

Self Cite

View full text Add to dashboard Cite

show abstract

“…Another direction forward is to develop creation and annihilation operators within nondiagonalizable dynamics. In the study of complex stochastic information processing, for example, this would allow analytic study of infinitememory processes generated by, say, stochastic pushdown and counter automata [51,[80][81][82]. In a physical context, such operators may aid in the study of open quantum field theories.…”

Section: Discussionmentioning

confidence: 99%

Beyond the spectral theorem: Spectrally decomposing arbitrary functions of nondiagonalizable operators

Riechers

Crutchfield

2018

AIP Advances

Self Cite

View full text Add to dashboard Cite

Nonlinearities in finite dimensions can be linearized by projecting them into infinite dimensions. Unfortunately, often the linear operator techniques that one would then use simply fail since the operators cannot be diagonalized. This curse is well known. It also occurs for finite-dimensional linear operators. We circumvent it by developing a meromorphic functional calculus that can decompose arbitrary functions of nondiagonalizable linear operators in terms of their eigenvalues and projection operators. It extends the spectral theorem of normal operators to a much wider class, including circumstances in which poles and zeros of the function coincide with the operator spectrum. By allowing the direct manipulation of individual eigenspaces of nonnormal and nondiagonalizable operators, the new theory avoids spurious divergences. As such, it yields novel insights and closed-form expressions across several areas of physics in which nondiagonalizable dynamics are relevant, including memoryful stochastic processes, open nonunitary quantum systems, and far-from-equilibrium thermodynamics.The technical contributions include the first full treatment of arbitrary powers of an operator. In particular, we show that the Drazin inverse, previously only defined axiomatically, can be derived as the negative-one power of singular operators within the meromorphic functional calculus and we give a general method to construct it. We provide new formulae for constructing projection operators and delineate the relations between projection operators, eigenvectors, and generalized eigenvectors.By way of illustrating its application, we explore several, rather distinct examples. First, we analyze stochastic transition operators in discrete and continuous time. Second, we show that nondiagonalizability can be a robust feature of a stochastic process, induced even by simple counting. As a result, we directly derive distributions of the Poisson process and point out that nondiagonalizability is intrinsic to it and the broad class of hidden semi-Markov processes. Third, we show that the Drazin inverse arises naturally in stochastic thermodynamics and that applying the meromorphic functional calculus provides closed-form solutions for the dynamics of key thermodynamic observables. Fourth, we show that many memoryful processes have power spectra indistinguishable from white noise, despite being highly organized. Nevertheless, whenever the power spectrum is nontrivial, it is a direct signature of the spectrum and projection operators of the process' hidden linear dynamic, with nondiagonalizable subspaces yielding qualitatively distinct line profiles. Finally, we draw connections to the Ruelle-Frobenius-Perron and Koopman operators for chaotic dynamical systems. PACS numbers: 02.50.-r 05.45.Tp 02.50.Ey 02.50.Ga... the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.A. Einstein [1, p. 165]

show abstract

“…The ergodic decomposition of the entropy rate ( 45 ) states that a stationary process is asymptotically deterministic, i.e.,

, if and only if almost all its ergodic components are asymptotically deterministic, i.e.,

almost surely. In contrast, the ergodic decomposition of the excess entropy (46) states that a stationary process is infinitary, i.e.,

, if some of its ergodic components are infinitary, i.e.,

with a nonzero probability, or if

, i.e., if the process is strongly nonergodic in particular, see [ 14 , 15 ].…”

Section: Applicationsmentioning

confidence: 99%

Approximating Information Measures for Fields

Dębowski

2020

Entropy

View full text Add to dashboard Cite

We supply corrected proofs of the invariance of completion and the chain rule for the Shannon information measures of arbitrary fields, as stated by Dębowski in 2009. Our corrected proofs rest on a number of auxiliary approximation results for Shannon information measures, which may be of an independent interest. As also discussed briefly in this article, the generalized calculus of Shannon information measures for fields, including the invariance of completion and the chain rule, is useful in particular for studying the ergodic decomposition of stationary processes and its links with statistical modeling of natural language.

show abstract

Signatures of infinity: Nonergodicity and resource scaling in prediction, complexity, and learning

Cited by 13 publications

References 50 publications

Statistical signatures of structural organization: The case of long memory in renewal processes

Statistical signatures of structural organization: The case of long memory in renewal processes

Beyond the spectral theorem: Spectrally decomposing arbitrary functions of nondiagonalizable operators

Approximating Information Measures for Fields

Contact Info

Product

Resources

About