An Information-Theoretic Primer on Complexity, Self-Organization, and Emergence Periodicals, Inc. Complexity 15: 11-28, 2009 Key Words: complexity; information theory; self-organization; emergence; predictive information; excess entropy; entropy rate; assortativeness; predictive efficiency; adaptation
Complex Systems Science aims to understand concepts like complexity, selforganization, emergence and adaptation, among others. The inherent fuzziness in complex systems definitions is complicated by the unclear relation among these central processes: does self-organisation emerge or does it set the preconditions for emergence? Does complexity arise by adaptation or is complexity necessary for adaptation to arise? The inevitable consequence of the current impasse is miscommunication among scientists within and across disciplines. We propose a set of concepts, together with their possible information-theoretic interpretations
INTRODUCTIONC omplex Systems Science studies general phenomena of systems comprised of many simple elements interacting in a nontrivial fashion. Currently, fuzzy quantifiers like "many" and "nontrivial" are inevitable. "Many" implies a number large enough so that no individual component/feature predominates the dynamics of the system, but not so large that features are completely irrelevant. Interactions need to be "nontrivial" so that the degrees of freedom are suitably reduced, but not constraining to the point that the arising structure possesses no further degree of freedom. Crudely put, systems with a huge number of components interacting trivially are explained by statistical mechanics, and systems with precisely defined and constrained interactions are the concern of fields like chemistry and engineering. In so far as the domain of complex systems science overlaps these fields, it contributes insights when the classical assumptions are violated.
This article was submitted as an invited paper resulting from the "UnderstandingIt is unsurprising that a similar vagueness afflicts the discipline itself, which notably lacks a common formal framework for analysis. There are a number of reasons for this. Because complex systems science is broader than physics, biology, sociology, ecology, or economics, its foundations cannot be reduced to a single discipline. Furthermore, systems which lie in the gap between the "very large" and the "fairly small" cannot be easily modeled with traditional mathematical techniques.Initially setting aside the requirement for formal definitions, we can summarize our general understanding of complex systems dynamics as follows:(1) complex systems are "open," and receive a regular supply of energy, information, and/or matter from the environment;(2) a large, but not too large, ensemble of individual components interact in a nontrivial fashion; in others words, studying the system via statistical mechanics would miss important properties brought about by interactions; (3) the nontrivial interactions result in internal constraints, leading to symmetry breaking in th...