Reducing Dimensionality in Multiple Instance Learning with a Filter Method

Abstract

Abstract. In this article, we describe a feature selection algorithm which can automatically find relevant features for multiple instance learning. Multiple instance learning is considered an extension of traditional supervised learning where each example is made up of several instances and there is no specific information about particular instance labels. In this scenario, traditional supervised learning can not be applied directly and it is necessary to design new techniques. Our approach is based on principles of the well-known Relief-F algorithm which is extended to select features in this new learning paradigm by modifying the distance, the difference function and computation of the weight of the features. Four different variants of this algorithm are proposed to evaluate their performance in this new learning framework. Experiment results using a representative number of different algorithms show that predictive accuracy improves significantly when a multiple instance learning classifier is learnt on the reduced data set.

Class-Dependent Dissimilarity Measures for Multiple Instance Learning

“…For MIL, the only example of using a class-dependent dissimilarity we are aware of is from [20]. Here, bag dissimilarities are used for feature selection.…”

“…Ideally, we would want to have the class information of both bags when determining their dissimilarity (e.g. using the overall minimum distance for two positive bags, as in [20], but for classification purposes, it is obvious that only the labels of the prototypes are available. For positive prototypes, we want to find out something about the presence of a concept in the test bag (denoted by B), i.e.…”

Feature selection is the ReliefF for multiple instance learning

“…The purpose of supervised learning is to learn to classify a new instance. In many applications, data, which is the subject of analysis and processing in data mining, is multidimensional, and presented by a number of features 1 . The so-called "curse of dimensionality" pertinent to many learning algorithms, denotes the drastic increase in computational complexity and classification error with data having a great number of dimensions.…”

“…In classification with multiple instance data, each example or pattern (often called bag) consists of a variable set of instances where you know the label of the example but there is no information about the labels of particular instances. This peculiarity introduces an additional challenge because the values of class labels with respect to particular instances that 1 We use the terms attributes and features as synonyms in this paper.…”

“…The preliminary study of this approach was presented in [1]. ReliefF-MI is based on the ideas of Relief [2], one of the state-of-the-art approaches for filter-based feature selection, which has been well studied and adopted as for classification (Relief-F [3]) as for regression (RRelief-F [4]) problem formulations.…”