Information, noise, coding, modulation: What about the brain?

Abstract

At the microscopic level, the brain is fundamentally a matter of physics and chemistry, as all the components of the universe are. At the macroscopic scale, behavior, psychology and affects are the main dimensions of its life. To convert atoms and molecules into intelligence, some kind of information has to be fixed in the grey matter of the cerebral cortex. The way this "mental information" is materialized and processed is still an enigma, probably the most puzzling problem addressed to science nowadays. At this mesoscopic level of the brain functioning, the concepts to consider are likely the same as those considered in communication and information theory, mainly information, noise, coding and modulation. This paper proposes some ideas that could help understand some features of the brain in an information-processing perspective.

Digital computing through randomness and order in neural networks

“…Some other neural architectures have been proposed recently to incorporate discreteness and digital capabilities. Instances have been proposed by Berrou et al ( 77 ) focusing on the problem of memory capacity and organization, while Graves and colleagues ( 78 , 79 ) investigated their computer-like features.…”

“…The neural network of Berrou et al ( 77 ) borrows the technique from telecommunication networks, showing high memory capacity and sparsity, but its use in real case problems and its computational efficacy in real time have not been investigated. Additionally, Graves’ neural Turing machine and differentiable neural computer (DNC) show the computational capabilities of conventional computers with the use of neuronal pointers and access to external memory.…”

On Neural Associative Memory Structures: Storage and Retrieval of Sequences in a Chain of Tournaments

“…Furthermore, they were implemented on a general-purpose graphical processing unit (GPU) (Yao, Gripon, & Rabbat, 2014), in 65-nm CMOS (Larras, Chollet, Lahuec, Seguin, & Arzel, 2018), and in distributed smart sensor architectures (Larras & Frappé, 2020). Therefore, CCN models can be referred to as an important brain-inspired memory system (Berrou, Dufor, Gripon, & Jiang, 2014) that became a basis for a wide range of research in associative memory models.…”

Sparse associative memory based on contextual code learning for disambiguating word senses

“…The ideas that are presented in this paper with a particular application to contextual word representations are inspired by recent studies about physicochemical properties of the brain. As explained in [8], the brain is a very noisy medium. More precisely, according to [9] and [10] respectively, neurons are subject to high erasure effects on their inputs (neurotransmitters are temporarily unavailable in a synapse to respond properly to an incoming spike) and are also the source of numerous spurious (non stimulated) spikes.…”