Efficient coding explains the universal law of generalization in human perception

Abstract

Perceptual generalization and discrimination are fundamental cognitive abilities. For example, if a bird eats a poisonous butterfly, it will learn to avoid preying on that species again by generalizing its past experience to new perceptual stimuli. In cognitive science, the "universal law of generalization" seeks to explain this ability and states that generalization between stimuli will follow an exponential function of their distance in "psychological space." Here, I challenge existing theoretical explanations for the universal law and offer an alternative account based on the principle of efficient coding. I show that the universal law emerges inevitably from any information processing system (whether biological or artificial) that minimizes the cost of perceptual error subject to constraints on the ability to process or transmit information.

Efficient Coding in the Economics of Human Brain Connectomics

“…In developing an efficient diffusion model of communication, we introduced the notion of compression efficiency, which describes the prioritization of either communication fidelity or lossy compression in structural networks. Five predictions of rate-distortion theory and information diffusion adapted from prior literature corroborated our findings, supporting the validity of 20 compression efficiency [29,27,25]. Broadly, our work advances the study of brain network communication efficiency, information integration, and neural noise by reframing brain network communication as diffusing messages governed by rate-distortion and efficient coding theories.…”

“…To evaluate the roles of resource efficiency and rate-distortion theory in the brain, we assess five previously published predictions of rate-distortion theory and information diffusion ( Figure 1C) [25,29]. The first prediction of rate-distortion theory is that communication systems should produce an information rate that is an exponential function of distortion.…”

“…To operationalize metabolic expenditure, we use ASL, which measures cerebral blood flow (CBF) and is correlated with glucose expenditure and ATP consumption ( Figure 1B) [24]. We join work modeling efficient coding with rate-distortion theory [25], a branch of information theory that provides the mathematical foundations of lossy compression [26]. By assuming that the minimal amount of noise is achieved by signals that diffuse along shortest paths [12,14,16], we calculate the optimal rate of signal transmission to communicate between brain regions with an expected transmission fidelity in the capacity-limited structural network.…”

“…To evaluate the validity of our efficient diffusion model, we assess five published predictions of rate-distortion theory and information diffusion ( Figure 1C) [27,25]. As we will describe in detail, hypotheses of information diffusion models posit how network structure guides propagating signals in support of metabolic efficiency, transmission fidelity, and information integration [28,12,13,29].…”

Generalization as diffusion: human function learning on graphs

“…More broadly, both the RBF and diffusion kernel can be understood as instantiations of Shepard's (1987) "universal law of generalization" in a function learning domain, by expressing generalization as an exponentially decaying function of the distance between two stimuli. Shepard famously proposed that the law of generalization should be the first law of psychology, while recent work has further entrenched it in fundamental properties of efficient coding (Sims, 2018) and measurement invariance (Frank, 2018).…”