Scalable Approximate FRNN-OWA Classification

Abstract

Fuzzy Rough Nearest Neighbour classification with Ordered Weighted Averaging operators (FRNN-OWA) is an algorithm that classifies unseen instances according to their membership in the fuzzy upper and lower approximations of the decision classes. Previous research has shown that the use of OWA operators increases the robustness of this model. However, calculating membership in an approximation requires a nearest neighbour search. In practice, the query time complexity of exact nearest neighbour search algorithms in more than a handful of dimensions is near-linear, which limits the scalability of FRNN-OWA. Therefore, we propose approximate FRNN-OWA, a modified model that calculates upper and lower approximations of decision classes using the approximate nearest neighbours returned by Hierarchical Navigable Small Worlds (HNSW), a recent approximative nearest neighbour search algorithm with logarithmic query time complexity at constant near-100% accuracy. We demonstrate that approximate FRNN-OWA is sufficiently robust to match the classification accuracy of exact FRNN-OWA while scaling much more efficiently. We test four parameter configurations of HNSW, and evaluate their performance by measuring classification accuracy and construction and query times for samples of various sizes from three large datasets. We find that with two of the parameter configurations, approximate FRNN-OWA achieves near-identical accuracy to exact FRNN-OWA for most sample sizes within query times that are up to several orders of magnitude faster.

Adapting Fuzzy Rough Sets for Classification with Missing Values

“…For crisp sets C, the choice of t-norm in (4) becomes void. In line with previous work [18] we use linearly decreasing weights We evaluate performance with the mean Area Under the Receiver Operator Curve (AUROC) across 5-fold cross-validation. For multi-class datasets, we use the extension of AUROC by Hand & Till (2001) [11].…”

Fuzzy-Rough Nearest Neighbour Approaches for Emotion Detection in Tweets

“…In this paper, we will use the fuzzy rough nearest neighbour (FRNN) classification algorithm originally proposed in [10], and refined later with Ordered Weighted Average (OWA) operators [3,12].…”

“…Our purpose in this paper is to explore the efficiency of the fuzzy-rough nearest neighbour (FRNN) classifier [10] and its extensions based on ordered weighted average (OWA) operators [3,12] for this task. The motivation behind the usage of FRNN is to investigate the potential of relatively simple and transparent instance-based methods for the emotion detection task, in comparison with the black-box solutions offered by deep learning approaches.…”

fuzzy-rough-learn 0.1: A Python Library for Machine Learning with Fuzzy Rough Sets

“…Given a new instance y, we obtain class scores by calculating the membership degree of y in the upper and lower approximations of each decision class and taking the mean. This implementation uses OWA weights, but limits their application to the k nearest neighbours of each class, as suggested by [8] (Table 3).…”

“…Similarly, users may customise the nearest neighbour search algorithm that is used in all classes except FRFS by defining their own subclass of NNSearch. For example, by choosing an approximative nearest neighbour search like Hierarchical Navigable Small World [9], we obtain Approximate FRNN [8].…”