Lifelong Learning Starting from Zero

Abstract

We present a deep neural-network model for lifelong learning inspired by several forms of neuroplasticity. The neural network develops continuously in response to signals from the environment. In the beginning the network is a blank slate with no nodes at all. It develops according to four rules: (i) expansion, which adds new nodes to memorize new input combinations; (ii) generalization, which adds new nodes that generalize from existing ones; (iii) forgetting, which removes nodes that are of relatively little use; and (iv) backpropagation, which fine-tunes the network parameters. We analyze the model from the perspective of accuracy, energy efficiency, and versatility and compare it to other network models, finding better performance in several cases.

Error driven synapse augmented neurogenesis

“…There is very limited work on neurogenesis that does not rely on a seed network, the best example comes from Strannegård et al ( 2019 ) in the paper Lifelong Learning Starting From Zero . They begin with an empty network and add neurons in response to errors, unlike the similar work of Eriksson and Westlund Gotby ( 2019 ) in which neurons are added for unrepresented states.…”