Incremental Learning with Self-labeling of Incoming High-dimensional Data

Abstract

Many incoming data chunks are being produced each day continuously at high speed with soaring dimensionality, and in most cases, these chunks are unlabeled. Our study combines incremental learning with self-labeling to deal with these incoming data chunks. We first search for the best data dimensionality reduction algorithm, leading to the optimal low-dimensional space for all the incoming chunks. The incremental classifier is then adapted gradually with chunks that are optimally reduced and self-labeled. Using a highly-dimensional and multi-class dataset, we conduct several experiments to demonstrate our incremental learning approach's efficacy and compare it with incremental learning using human-annotated labels.