The Distributed Information Bottleneck reveals the explanatory structure of complex systems

Murphy, Kieran; Bassett, Dani S.

doi:10.48550/arxiv.2204.07576

Cited by 2 publications

(5 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They resonate with the conceptual spaces theory of Gärdenfors [81,82], as well as with the notion of information space [83]. Future work could also look into resonance between our and the recent deep learning driven "distributed information bottleneck" proposal [29] which also involves visual transformations in the case of images.…”

Section: A Window To Cultural Meaning Spacesupporting

confidence: 64%

“…Deep learning models have made headway in solving the latter among other object recognition tasks, but are not very good at explaining why or how they recognize something either (though cf. [29]).…”

Section: Family Resemblance and Connoisseurshipmentioning

confidence: 99%

“…The quantification of visual aesthetics and artistic expression goes back to Birkhoff [1] and Bense [2], inspiring several computational approaches [3][4][5][6][7][8][9][10][11][12][13][14][15][16]. Research drawing on information theory has shown repeatedly and in parallel that visual complexity can be estimated with some accuracy using compression algorithms such as zip or gif [17][18][19][20][21][22][23][24][25][26][27][28][29]. While some previous proposals were tested against perceptual human judgments, results diverge as to which compression algorithm or approach would be optimal.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Compression ensembles quantify aesthetic complexity and the evolution of visual art

et al. 2023

View full text Add to dashboard Cite

To the human eye, different images appear more or less complex, but capturing this intuition in a single aesthetic measure is considered hard. Here, we propose a computationally simple, transparent method for modeling aesthetic complexity as a multidimensional algorithmic phenomenon, which enables the systematic analysis of large image datasets. The approach captures visual family resemblance via a multitude of image transformations and subsequent compressions, yielding explainable embeddings. It aligns well with human judgments of visual complexity, and performs well in authorship and style recognition tasks. Showcasing the functionality, we apply the method to 125,000 artworks, recovering trends and revealing new insights regarding historical art, artistic careers over centuries, and emerging aesthetics in a contemporary NFT art market. Our approach, here applied to images but applicable more broadly, provides a new perspective to quantitative aesthetics, connoisseurship, multidimensional meaning spaces, and the study of cultural complexity.

show abstract

Section: A Window To Cultural Meaning Spacesupporting

confidence: 64%

Section: Family Resemblance and Connoisseurshipmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Compression ensembles quantify aesthetic complexity and the evolution of visual art

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Decomposing the information in a composite

depends upon its basis of measurements ( 52 ). In the study of complex systems, there can be multiple “natural” schemes of measuring a system state.…”

Section: Resultsmentioning

confidence: 99%

“…However, the IB generally has limited interpretability because the singular bottleneck occurs after processing the complete input, allowing the compression scheme to involve arbitrarily complex relationships between components of the input without penalty. Additionally, when the relationship between

and Y is deterministic (or nearly so), the entire spectrum of extracted information is trivially a copy of Y with added noise ( 52 , 61 , 62 ). The distribution of IBs across observables is critical to an interpretable information decomposition and to recovering an illuminating spectrum of extracted information.…”

Section: Discussionmentioning

confidence: 99%

Information decomposition in complex systems via machine learning

Murphy,

Bassett

2024

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

One of the fundamental steps toward understanding a complex system is identifying variation at the scale of the system’s components that is most relevant to behavior on a macroscopic scale. Mutual information provides a natural means of linking variation across scales of a system due to its independence of functional relationship between observables. However, characterizing the manner in which information is distributed across a set of observables is computationally challenging and generally infeasible beyond a handful of measurements. Here, we propose a practical and general methodology that uses machine learning to decompose the information contained in a set of measurements by jointly optimizing a lossy compression of each measurement. Guided by the distributed information bottleneck as a learning objective, the information decomposition identifies the variation in the measurements of the system state most relevant to specified macroscale behavior. We focus our analysis on two paradigmatic complex systems: a Boolean circuit and an amorphous material undergoing plastic deformation. In both examples, the large amount of entropy of the system state is decomposed, bit by bit, in terms of what is most related to macroscale behavior. The identification of meaningful variation in data, with the full generality brought by information theory, is made practical for studying the connection between micro- and macroscale structure in complex systems.

show abstract

The Distributed Information Bottleneck reveals the explanatory structure of complex systems

Cited by 2 publications

References 59 publications

Compression ensembles quantify aesthetic complexity and the evolution of visual art

Compression ensembles quantify aesthetic complexity and the evolution of visual art

Information decomposition in complex systems via machine learning

Contact Info

Product

Resources

About