Non-sufficient Memories That Are Sufficient for Prediction

Löhr, Wolfgang; Ay, Nihat

doi:10.1007/978-3-642-02466-5_25

Cited by 13 publications

(20 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This was previously shown in Ref. [4]. However, the question remained whether or not the Löhr model is minimal.…”

Section: Löhr Examplementioning

confidence: 61%

“…3. Reference [4] offers up a three-state HMM generator for this class, which we refer to as the Löhr model; see Fig. 4.…”

Section: Löhr Examplementioning

confidence: 99%

“…The generative machine offers more challenges, as it involves a nonconvex constrained minimization over high-dimensional spaces. While there are several known bounds on C g and restrictions on the construction of generative HMMs [3][4][5][6], they have received significantly less attention than the predictive case and, as a consequence, are markedly less well understood.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Prediction and generation of binary Markov processes: Can a finite-state fox catch a Markov mouse?

Ruebeck

James

Mahoney

et al. 2018

Chaos: An Interdisciplinary Journal of Nonlinear Science

View full text Add to dashboard Cite

Understanding the generative mechanism of a natural system is a vital component of the scientific method. Here, we investigate one of the fundamental steps toward this goal by presenting the minimal generator of an arbitrary binary Markov process. This is a class of processes whose predictive model is well known. Surprisingly, the generative model requires three distinct topologies for different regions of parameter space. We show that a previously proposed generator for a particular set of binary Markov processes is, in fact, not minimal. Our results shed the first quantitative light on the relative (minimal) costs of prediction and generation. We find, for instance, that the difference between prediction and generation is maximized when the process is approximately independently, identically distributed.

show abstract

“…This was previously shown in Ref. [4]. However, the question remained whether or not the Löhr model is minimal.…”

Section: Löhr Examplementioning

confidence: 61%

“…3. Reference [4] offers up a three-state HMM generator for this class, which we refer to as the Löhr model; see Fig. 4.…”

Section: Löhr Examplementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Prediction and generation of binary Markov processes: Can a finite-state fox catch a Markov mouse?

Ruebeck

James

Mahoney

et al. 2018

Chaos: An Interdisciplinary Journal of Nonlinear Science

View full text Add to dashboard Cite

show abstract

“…The methods and results here should also be extended to analyze classical generative models which, in many ways, bear resemblances in their functionality to the quantum models [34][35][36]. These drop the requirement of unifilarity, similar to how the quantum models relax the notion of orthogonality.…”

Section: Discussionmentioning

confidence: 99%

Strong and Weak Optimizations in Classical and Quantum Models of Stochastic Processes

Loomis

Crutchfield

2019

J Stat Phys

View full text Add to dashboard Cite

Among the predictive hidden Markov models that describe a given stochastic process, the -machine is strongly minimal in that it minimizes every Rényi-based memory measure. Quantum models can be smaller still. In contrast with the -machine's unique role in the classical setting, however, among the class of processes described by pure-state hidden quantum Markov models, there are those for which there does not exist any strongly minimal model. Quantum memory optimization then depends on which memory measure best matches a given problem circumstance. PACS numbers: 05.45.-a 89.75.Kd 89.70.+c 05.45.Tp arXiv:1808.08639v1 [quant-ph]

show abstract

“…The first part of the paper, Sections 2 and 3, respectively, is a slightly adapted and shortened version of [10]. The proofs of the propositions in these sections can be found in the appendix of [9].…”

Section: Outlinementioning

confidence: 99%

Predictive models and generative complexity

Löhr

2012

J Syst Sci Complex

Self Cite

View full text Add to dashboard Cite

The causal states of computational mechanics define the minimal sufficient memory for a given discrete stationary stochastic process. Their entropy is an important complexity measure called statistical complexity (or true measure complexity). They induce the ε-machine, which is a hidden Markov model (HMM) generating the process. But it is not the minimal one, although generative HMMs also have a natural predictive interpretation. This paper gives a mathematical proof of the idea that the ε-machine is the minimal HMM with an additional (partial) determinism condition. Minimal internal state entropy of a generative HMM is in analogy to statistical complexity called generative complexity. This paper also shows that generative complexity depends on the process in a nice way. It is, as a function of the process, lower semi-continuous (w.r.t. weak- * topology), concave, and behaves nice under ergodic decomposition of the process.

show abstract

Non-sufficient Memories That Are Sufficient for Prediction

Cited by 13 publications

References 6 publications

Prediction and generation of binary Markov processes: Can a finite-state fox catch a Markov mouse?

Prediction and generation of binary Markov processes: Can a finite-state fox catch a Markov mouse?

Strong and Weak Optimizations in Classical and Quantum Models of Stochastic Processes

Predictive models and generative complexity

Contact Info

Product

Resources

About