BinPlay: A Binary Latent Autoencoder for Generative Replay Continual Learning

Deja, Kamil Rafał; Wawrzyński, Paweł; Marczak, Daniel; Masarczyk, Wojciech; Trzciński, Tomasz Piotr

doi:10.1109/ijcnn52387.2021.9534171

Cited by 3 publications

(7 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our method builds on our previous work, and we utilize the BinPlay [9] model for our experiments. BinPlay is an autoencoder whose main objective is to find a qualitypreserving encoding of past samples into precomputed binary codes living in the autoencoder's binary latent space.…”

Section: Methodsmentioning

confidence: 99%

“…In our previous work [9], we propose BinPlay, an architecture for generative rehearsal and continual learning. BinPlay, rather than being trained to generate data similar to the observed samples, learns to reconstruct only the observed examples.…”

Section: Related Workmentioning

confidence: 99%

“…In the following sections, whenever the autoencoder is in use, we apply the architecture introduced by [9] with an adjusted size of the latent space for each dataset 1 As a measure of performance, we report the average accuracy of the global classifier on the whole test set after finishing the training sequence.…”

Section: B Main Experimentsmentioning

confidence: 99%

“…The approach to address this problem is generative rehearsal [6]- [9] which utilizes a single generative model to rehearse past data and retrain a model with a combination of current and regenerated samples. Memory consumption of such a method is fixed throughout the whole training.…”

Section: Introductionmentioning

confidence: 99%

“…Our method, based on our previous method called Bin-Play [9], allocates incoming data samples in generative models of limited capacity, as depicted in Fig 1 . In our approach, the i-th model stores exactly 2 i samples. Therefore the total number of utilized models equals at most log 2 n, where n is the number of data samples seen so far.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Logarithmic Continual Learning

et al. 2022

Self Cite

View full text Add to dashboard Cite

We introduce a neural network architecture that logarithmically reduces the number of selfrehearsal steps in the generative rehearsal of continually learned models. In continual learning (CL), training samples come in subsequent tasks, and the trained model can access only a current task. Contemporary CL methods employ generative models to replay previous samples and train them recursively with a combination of current and regenerated past data. This recurrence leads to superfluous computations as the same past samples are regenerated after each task, and the reconstruction quality successively degrades. In this work, we address these limitations and propose a new generative rehearsal architecture that requires, at most, a logarithmic number of retraining sessions for each sample. Our approach leverages the allocation of past data in a set of generative models such that most of them do not require retraining after a task. The experimental evaluation of our logarithmic continual learning approach shows the superiority of our method with respect to the state-of-the-art generative rehearsal methods.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%