Information Transfer in Canonical Systems

Bossomaier, Terry; Barnett, Lionel; Harré, Michael; Lizier, Joseph T.

doi:10.1007/978-3-319-43222-9_5

Cited by 34 publications

(93 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the first instance, this simplifies eqs. (21) and (22) (see also [40,[54][55][56][57]). Returning, for continuity, to an expression of paths, x, we can represent any path containing N x spikes starting at time t 0 as x t t0 ≡ {t, {t} …”

Section: Application To Spike Trainsmentioning

confidence: 94%

“…(1) represents a rate of a transfer of information per discretized time step [38]. Consequently, without such a fundamental temporal discretization we must initially define a transfer entropy rate in Proposition 1 (see also [39][40][41][42]). We emphasize that this naturally leads to integrated quantities, in the form of functionals of realized paths, which we introduce subsequently (Proposition 2).…”

Section: B Continuous Time Formalismmentioning

confidence: 99%

See 1 more Smart Citation

Transfer entropy in continuous time, with applications to jump and neural spiking processes

2017

Self Cite

View full text Add to dashboard Cite

Transfer entropy has been used to quantify the directed flow of information between source and target variables in many complex systems. While transfer entropy was originally formulated in discrete time, in this paper we provide a framework for considering transfer entropy in continuous time systems, based on Radon-Nikodym derivatives between measures of complete path realizations. To describe the information dynamics of individual path realizations, we introduce the pathwise transfer entropy, the expectation of which is the transfer entropy accumulated over a finite time interval. We demonstrate that this formalism permits an instantaneous transfer entropy rate. These properties are analogous to the behavior of physical quantities defined along paths such as work and heat. We use this approach to produce an explicit form for the transfer entropy for pure jump processes, and highlight the simplified form in the specific case of point processes (frequently used in neuroscience to model neural spike trains). Finally, we present two synthetic spiking neuron model examples to exhibit the pertinent features of our formalism, namely, that the information flow for point processes consists of discontinuous jump contributions (at spikes in the target) interrupting a continuously varying contribution (relating to waiting times between target spikes). Numerical schemes based on our formalism promise significant benefits over existing strategies based on discrete time formalisms.

show abstract

Section: Application To Spike Trainsmentioning

confidence: 94%

Section: B Continuous Time Formalismmentioning

confidence: 99%

Transfer entropy in continuous time, with applications to jump and neural spiking processes

2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…The concept of information processing presented herein is adopted from information dynamics [29][30][31][32], a formalism that quantifies computation in dynamical systems using methods from information theory. In information dynamics, Schreiber's transfer entropy [27] is identified with the information processed between two elements of a dynamical system.…”

Section: Intrinsic Complexitymentioning

confidence: 99%

“…We find very similar values of TE for = 4 and 6 and observe a considerable decrease in TE for < 4 and > 6 (cf. Section 4.2 of [32]), relative to the value of TE for = 5. We therefore consider = 5 as the optimal value of that properly captures the past history of for the results presented here (see Section 3 for a visual explanation of why ∼ 5 represents the optimal history length).…”

Section: Intrinsic Complexitymentioning

confidence: 99%

“…Smaller and larger values of would not be able to resolve this lattice size independent feature of Class I 3 rules [32] (Figure 4). 0 1 3 4 5 8 9 1 2 1 3 18 19 23 25 26 28 29 32 33 35 36 37 40 41 44 50 51 57 58 62 72 73 76 77 78 94 104 108 128 130 132 134 136 140 146 152 154 156 160 162 164 178 200 204 …”

Section: Inherited Complexity and Information-based Classificationmentioning

confidence: 99%

See 1 more Smart Citation

An Information-Based Classification of Elementary Cellular Automata

Borriello

Walker

2017

Complexity

View full text Add to dashboard Cite

We propose a novel, information-based classification of elementary cellular automata. The classification scheme proposed circumvents the problems associated with isolating whether complexity is in fact intrinsic to a dynamical rule, or if it arises merely as a product of a complex initial state. Transfer entropy variations processed by cellular automata split the 256 elementary rules into three information classes, based on sensitivity to initial conditions. These classes form a hierarchy such that coarse-graining transitions observed among elementary rules predominately occur within each information-based class or, much more rarely, down the hierarchy.

show abstract

Information dynamics in neuromorphic nanowire networks

Zhu

Hochstetter

Loeffler

et al. 2021

Sci Rep

Self Cite

View full text Add to dashboard Cite

Neuromorphic systems comprised of self-assembled nanowires exhibit a range of neural-like dynamics arising from the interplay of their synapse-like electrical junctions and their complex network topology. Additionally, various information processing tasks have been demonstrated with neuromorphic nanowire networks. Here, we investigate the dynamics of how these unique systems process information through information-theoretic metrics. In particular, Transfer Entropy (TE) and Active Information Storage (AIS) are employed to investigate dynamical information flow and short-term memory in nanowire networks. In addition to finding that the topologically central parts of networks contribute the most to the information flow, our results also reveal TE and AIS are maximized when the networks transitions from a quiescent to an active state. The performance of neuromorphic networks in memory and learning tasks is demonstrated to be dependent on their internal dynamical states as well as topological structure. Optimal performance is found when these networks are pre-initialised to the transition state where TE and AIS are maximal. Furthermore, an optimal range of information processing resources (i.e. connectivity density) is identified for performance. Overall, our results demonstrate information dynamics is a valuable tool to study and benchmark neuromorphic systems.

show abstract

Information Transfer in Canonical Systems

Cited by 34 publications

References 0 publications

Transfer entropy in continuous time, with applications to jump and neural spiking processes

Transfer entropy in continuous time, with applications to jump and neural spiking processes

An Information-Based Classification of Elementary Cellular Automata

Information dynamics in neuromorphic nanowire networks

Contact Info

Product

Resources

About