Neural networks: learning and forgetting

Stochastic Consolidation of Lifelong Memory

“…Without rehearsals, our model is similar to previous models of associative memory with forgetting [8,10,11], in which memory efficacies decay exponentially with age, A l (t) = exp(−(t − l)/τ ) (Fig. 1a).…”

“…Incorporating synaptic decay into the circuit enables continual learning, such that at any point in time recent memories are stable. However, the predicted forgetting curves exhibit a critical memory age, all memories newer than some age are almost perfectly retrievable, while all older ones are destroyed [8][9][10][11][12][13]. This is in contrast to the gracefully decaying forgetting curves in humans.…”

“…Synaptic dynamics are governed by three processes: Deterministic exponential synaptic decay [8,11] (first term in eq. ( 1)), Hebbian learning [39] of new memories (second term in eq.…”

“…To assess the efficiency of information storage in the network it is important to evaluate the dependence of the capacity on the network size, N . In previous 'pure forgetting' models [8,10] the synaptic decay time was assumed to scale linearly with N , resulting in memory capacity t 0 which is proportional to N . The same holds for our model.…”

Stochastic consolidation of lifelong memory

“…Incorporating synaptic decay into the circuit enables continual learning, such that at any point in time recent memories are stable. However, the predicted forgetting curves exhibit a critical memory age, all memories newer than some age are almost perfectly retrievable, while all older ones are destroyed (a palimpsestic behavior-old information is deleted in favor of new information) 8 – 13 . This is in contrast to the gracefully decaying forgetting curves in humans.…”