An information‐theoretic primer on complexity, self‐organization, and emergence

Abstract

An Information-Theoretic Primer on Complexity, Self-Organization, and Emergence Periodicals, Inc. Complexity 15: 11-28, 2009 Key Words: complexity; information theory; self-organization; emergence; predictive information; excess entropy; entropy rate; assortativeness; predictive efficiency; adaptation Complex Systems Science aims to understand concepts like complexity, selforganization, emergence and adaptation, among others. The inherent fuzziness in complex systems definitions is complicated by the unclear relation among these central processes: does self-organisation emerge or does it set the preconditions for emergence? Does complexity arise by adaptation or is complexity necessary for adaptation to arise? The inevitable consequence of the current impasse is miscommunication among scientists within and across disciplines. We propose a set of concepts, together with their possible information-theoretic interpretations INTRODUCTIONC omplex Systems Science studies general phenomena of systems comprised of many simple elements interacting in a nontrivial fashion. Currently, fuzzy quantifiers like "many" and "nontrivial" are inevitable. "Many" implies a number large enough so that no individual component/feature predominates the dynamics of the system, but not so large that features are completely irrelevant. Interactions need to be "nontrivial" so that the degrees of freedom are suitably reduced, but not constraining to the point that the arising structure possesses no further degree of freedom. Crudely put, systems with a huge number of components interacting trivially are explained by statistical mechanics, and systems with precisely defined and constrained interactions are the concern of fields like chemistry and engineering. In so far as the domain of complex systems science overlaps these fields, it contributes insights when the classical assumptions are violated. This article was submitted as an invited paper resulting from the "UnderstandingIt is unsurprising that a similar vagueness afflicts the discipline itself, which notably lacks a common formal framework for analysis. There are a number of reasons for this. Because complex systems science is broader than physics, biology, sociology, ecology, or economics, its foundations cannot be reduced to a single discipline. Furthermore, systems which lie in the gap between the "very large" and the "fairly small" cannot be easily modeled with traditional mathematical techniques.Initially setting aside the requirement for formal definitions, we can summarize our general understanding of complex systems dynamics as follows:(1) complex systems are "open," and receive a regular supply of energy, information, and/or matter from the environment;(2) a large, but not too large, ensemble of individual components interact in a nontrivial fashion; in others words, studying the system via statistical mechanics would miss important properties brought about by interactions; (3) the nontrivial interactions result in internal constraints, leading to symmetry breaking in th...

The World as Evolving Information

“…Another benefit of using information as a basic descriptor for our world is that the concept is well studied and formal methods have already been developed (Cover and Thomas, 2006;Prokopenko et al, 2007), as well as its philosophical implications have been discussed (Floridi, 2003). Thus, there is no need to develop a new formalism, since information theory is well established.…”

“…The amount of information required to describe a process, system, object, or agent determines its complexity (Prokopenko et al, 2007). According to our current knowledge, during the evolution of our universe there has been a shift from simple information towards more complex information (Adami, 2002) (the information of an atom is less complex than that of a molecule, than that of a cell, than that of a multicellular organism, etc.).…”

“…amount (Prokopenko et al, 2007). However, more complex information will require more complex agents to perceive, act on, and propagate it.…”

The “weight” of models and complexity

“…A variety of complexity measures has been proposed in this journal including network complexity [37,38], complexity as the difficulty of performing relevant tasks [39], complexity in information theory [40,41], and many measures discussed by Murray Gell-Mann [42]. This work is different in two ways.…”

The complexity of partition tasks

“…Significant work regarding complexity has been carried out in different fields such as physics [24,27,2], biology [8,25], computer science [12,16,23] and social sciences [9,11,7,4]. In many of these cases, the framework presented in this paper can be useful since the P T problem class is very general.…”