Learning and organization of memory for evolving patterns

Abstract

Storing memory for molecular recognition is an efficient strategy for responding to external stimuli. Biological processes use different strategies to store memory. In the olfactory cortex, synaptic connections form when stimulated by an odor, and establish distributed memory that can be retrieved upon re-exposure. In contrast, the immune system encodes specialized memory by diverse receptors that recognize a multitude of evolving pathogens. Despite the mechanistic differences between the olfactory and the immune memory, these systems can still be viewed as different information encoding strategies. Here, we present a theoretical framework with artificial neural networks to characterize optimal memory strategies for both static and dynamic (evolving) patterns. Our approach is a generalization of the energy-based Hopfield model in which memory is stored as a network's energy minima. We find that while classical Hopfield networks with distributed memory can efficiently encode a memory of static patterns, they are inadequate against evolving patterns. To follow an evolving pattern, we show that a distributed network should use a higher learning rate, which in turn, can distort the energy landscape associated with the stored memory attractors. Specifically, narrow connecting paths emerge between memory attractors, leading to misclassification of evolving patterns. We demonstrate that compartmentalized networks with specialized subnetworks are the optimal solutions to memory storage for evolving patterns. We postulate that evolution of pathogens may be the reason for the immune system to encoded a focused memory, in contrast to the distributed memory used in the olfactory cortex that interacts with mixtures of static odors.

Risk-utility tradeoff shapes memory strategies for evolving patterns

“…, where ρ = (1 − 2µ) measures the similarity between evolved patterns. Interestingly, for Θ = 2 this model is equivalent to the energy function of the classical Hopfield network with Hebbian learning rule [9,10,12,13] (Appendix B). This correspondence enables us to simulate the memory repertoire efficiently and to test analytic predictions with numerical experiments; see Appendix C for numerical method.…”

“…This result is in line with the eigendecomposition analysis of the generalized Hopfield network in Appendix C of ref. [9].…”

“…One such example is the adaptive immune system in which memory can effectively recognize evolved variants of previously encountered pathogens [4][5][6][7][8]. In a recent work, we have demonstrated that distributed learning strategies, which are desirable for pattern recognition in the stationary setup, can fail to reliably learn and classify dynamically evolving patterns [9]. Specifically, we showed that to follow evolving patterns, an energy-based Hopfield-like neural network [10] should use a higher learning rate, which in turn, can distort the energy landscape associated with the stored memory attractors, leading to pattern misclassification.…”

“…Specifically, we showed that to follow evolving patterns, an energy-based Hopfield-like neural network [10] should use a higher learning rate, which in turn, can distort the energy landscape associated with the stored memory attractors, leading to pattern misclassification. To remedy this problem, we proposed compartmentalized networks as the optimal solutions to memory storage for evolving patterns [9].…”

“…Irrespective of the network structure, an increase in learning rate is necessary for a network to follow, recognize, and store effective memory of evolving patterns [9]. Increasing learning rate leads to a risky strategy, as the memory repertoire begins to reflect only the most recently encountered pattern while effectively destroying the memory of the prior encounters.…”