Enhancing Open-Set Recognition using Clustering-based Extreme Value Machine (C-EVM)

Abstract

In real-world deployments, machine learning applications find challenges when accessing everincreasing volumes of data -the real world is open and often presents data from classes not seen in training. Open-set recognition is a growing area of machine learning addressing such problems. This research work advances the state-of-the-art in open-set recognition, the Extreme Value Machine (EVM), with a novel clustering-based extension (C-EVM) during training to improve the end-to-end prediction performance. The C-EVM combines Density-based spatial clustering of applications with noise (DBSCAN)-based clustering with a novel Nearby Clusters (NC) algorithm during model fitting to reduce computation while improving accuracy. Our experiments show a statistically significant improvement of 5-10% in macro F1-score over the state-of-the-art EVM on open-set testing using the KDD CUP-99 data set. Past work on open-set recognition often traded improved open-set robustness for a decrease in closed-set accuracy, whereas C-EVM outperforms the EVM in both closed-set and open-set recognition. Testing on subsets of ImageNet-2012 with varying numbers of classes, the C-EVM statistically significantly outperforms EVM when using deep features. A parameterless Hierarchical DBSCAN (HDBSCAN)-based C-EVM variant is introduced as part of this work that scales well for large data sets. Finally, both EVM and C-EVM can operate as kernel-free incremental learners, enabling these open-set multi-class classifiers to be useful for streaming and big data applications.

Open-set face recognition with maximal entropy and Objectosphere loss

“…Vareto et al [22] combined hashing functions to set up a vote-list histogram. Some researchers have adopted one-vs-all SVM or PLS models [23,24] whereas others explored clustering techniques [25,26]. The aforementioned methods neither implement the entire closed-set identification pipeline nor comply with the requirements of real-time or real-world applications.…”

Exploring the Open World Using Incremental Extreme Value Machines

“…The EVM has achieved state-of-the-art results in intrusion detection [5] and open set face recognition [3]. The C-EVM [41] performs a clustering prior to the actual EVM fitting to reduce the dataset size. These centroids are then used to fit the EVM.…”

“…c) Relationship to Previous Works [10], [41]: Our weighted maximum K-set cover formulation in ( 6) - (8) generalizes the conventional set cover model reduction of Rudd et al [10]. To formulate [10] in our framework, we need to substitute Ψ i (x j ) and Ψ j (x i ) in (6) by I(θ i ) and I(θ j ), i. e., the indicator function of (4).…”

“…The C-EVM [41] uses class-wise DBSCAN clustering [44] and generates centroids from these clusters. This preconditioning reduces the training set size before the actual EVM is fitted to the centroids.…”

“…All EVMs use the same parameters: τ = 75 and α = 0.5. For the clustering in the C-EVM and C-iEVM, we adopt the parameters reported in [41]. Methods that employ a model reduction reduce the amount of EVs to K = 10.…”