Complexity, parallel computation and statistical physics

Machta, Jonathan

doi:10.1002/cplx.20125

Cited by 13 publications

(10 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This proposal, though consistent with the concept of depth introduced by Machta [4], has its own limitations, as we generally simulate approximate models of a system, rather than exact ones.…”

mentioning

confidence: 89%

A Self-Organization Model for Complex Computing and Communication Systems

Marinescu

Morrison

Chen

et al. 2008

2008 Second IEEE International Conference on Self-Adaptive and Self-Organizing Systems

View full text Add to dashboard Cite

We introduce a model of self-organizing systems inspired by biological and Physics metaphors. The motivation for the development of this model is twofold, it enables us to identify critical properties of a system and to understand the important metrics characterizing its qualitative and the quantitative behavior. It also enables us to develop a methodology to investigate these systems and to carry out comparative studies of alternative policies that govern system behavior. We illustrate the application of the model with a preliminary study of a system designed to encourage green computing and we present some preliminary simulation results of the model. I. MOTIVATIONEngineering modern computer and communication systems is increasingly more challenging because the systems are more complex and more difficult to manage and to use. The complexity of computer and communication systems is caused by several interrelated factors summarized in Figure 1. New applications are developed to satisfy the more diverse needs of an increasingly larger population of users. New devices such as sensors find a wide range of applications, and, in turn, trigger the development of new applications and attract new users. At the same time, the individual components of a system are more complex, for example, 2-4 core processors are ubiquitous now and will be replaced by multi-core systems with tens to hundreds of cores in the next few years. Connectivity and mobility, required by virtually all users of the systems, increase the complexity of the applications and of the computer and communication infrastructure. At the same time, physical limitations, such as heat removal, bandwidth availability, the capacity to store energy, and other factors place additional constrains upon system design. Last, but not least, we have finite resources and the optimization of resource consumption increases system complexity. . Larger segment of population using the systems New and more complex components Optimization of resource consumption New and more complex applications Complexity of computing and communication systems Interconnectivity + mobility Physical constraints Fig. 1. Some of the factors contributing to the complexity of modern computer and communication systems.While we have an intuitive notion of the system complexity, a rigorous definition that allows us to quantify and measure the complexity of a system is not universally accepted. The thermodynamic, von Neumann, and Shannon entropy are related to the number of states of a system, thus they reflect to some extent the system complexity. Kolmogorov complexity [1], [2] of an object is a measure of computational resources needed to specify the object. Using a Markovian assumption and a Kologorov complexity model, we argued that scheduling and resource management are more complex on a computational grid than on a service grid due to the finer granularity of resource allocation [3].We propose to uses the complexity of a program that simulates the system as a measure of complexity of the system; this...

show abstract

mentioning

confidence: 89%

A Self-Organization Model for Complex Computing and Communication Systems

Marinescu

Morrison

Chen

et al. 2008

2008 Second IEEE International Conference on Self-Adaptive and Self-Organizing Systems

View full text Add to dashboard Cite

show abstract

“…Systems with long histories (i.e., mechanisms of long-term memory) allow for the "emergence", or accumulation of physical properties in a growing space of otherwise highly unpredictable states. This idea has been captured intuitively through a complexity measure -algorithmic depth -which seeks to equate complexity with historical depth (Bennett, 1988;Machta, 2006). Hence complex systems are systems for which a full understanding requires a specification of a historical sequence.…”

Section: The Challenges and Character Of Biological Theorymentioning

confidence: 99%

The challenges and scope of theoretical biology

Krakauer

Collins

Erwin

et al. 2011

Journal of Theoretical Biology

View full text Add to dashboard Cite

“…On the other hand, note that there are a variety of other measures for what is known as algorithmic entropy, as, e.g., the description size of a minimal algorithm (or computer, or circuit) which is able to generate an instance of the problem under scrutiny [13][14][15]. However, such measures are often impractical when it comes to the analysis of large systems.…”

Section: Introductionmentioning

confidence: 99%

Analysis of the phase transition in the two-dimensional Ising ferromagnet using a Lempel-Ziv string-parsing scheme and black-box data-compression utilities

Melchert

Hartmann

2015

Phys. Rev. E

View full text Add to dashboard Cite

In this work we consider information-theoretical observables to analyze short symbolic sequences, comprising time-series that represent the orientation of a single spin in a 2D Ising ferromagnet on a square lattice of size L 2 = 128 2 , for different system temperatures T . The latter were chosen from an interval enclosing the critical point Tc of the model. At small temperatures the sequences are thus very regular, at high temperatures they are maximally random. In the vicinity of the critical point, nontrivial, long-range correlations appear. Here, we implement estimators for the entropy rate, excess entropy (i.e. "complexity") and multi-information. First, we implement a Lempel-Ziv string parsing scheme, providing seemingly elaborate entropy rate and multi-information estimates and an approximate estimator for the excess entropy. Furthermore, we apply easy-to-use black-box data compression utilities, providing approximate estimators only. For comparison and to yield results for benchmarking purposes we implement the information-theoretic observables also based on the well-established M -block Shannon entropy, which is more tedious to apply compared to the the first two "algorithmic" entropy estimation procedures. To test how well one can exploit the potential of such data compression techniques, we aim at detecting the critical point the 2D Ising ferromagnet. Among the above observables, the multi-information, which is known to exhibit an isolated peak at the critical point, is very easy to replicate by means of both efficient algorithmic entropy estimation procedures. Finally, we assess how good the various algorithmic entropy estimates compare to the more conventional block entropy estimates and illustrate a simple modification that yields enhanced results.

show abstract

Complexity, parallel computation and statistical physics

Cited by 13 publications

References 61 publications

A Self-Organization Model for Complex Computing and Communication Systems

A Self-Organization Model for Complex Computing and Communication Systems

The challenges and scope of theoretical biology

Analysis of the phase transition in the two-dimensional Ising ferromagnet using a Lempel-Ziv string-parsing scheme and black-box data-compression utilities

Contact Info

Product

Resources

About