Feature Construction Based on Closedness Properties Is Not That Simple

Abstract

Abstract. Feature construction has been studied extensively, including for 0/1 data samples. Given the recent breakthrough in closedness-related constraint-based mining, we are considering its impact on feature construction for classification tasks. We investigate the use of condensed representations of frequent itemsets (closure equivalence classes) as new features. These itemset types have been proposed to avoid set counting in difficult association rule mining tasks. However, our guess is that their intrinsic properties (say the maximality for the closed itemsets and the minimality for the δ-free itemsets) might influence feature quality. Understanding this remains fairly open and we discuss these issues thanks to itemset properties on the one hand and an experimental validation on various data sets on the other hand.

Application-Independent Feature Construction from Noisy Samples

“…In this paper, we investigate further the use of δ-freeness (and thus δ-closedness) when considering feature construction from noisy training data. From that perspective, it extends our previous work [11] which focussed on noise-free samples only. Our proposal can be summarized as follows.…”

“…Using such patterns is the core of our approach to robust feature construction. Following the proposals from, for instance, [10,11,12], we consider that attribute sets may be more relevant than single attributes for class discrimination. Then, pattern types based on the so-called closedness properties enable to avoid redundant features in an application-independent setting.…”

“…In pioneering studies, association rules were considered [13]. Since then, emerging patterns [14] (see [15] for a survey and [16] for a noise tolerant pattern-based approach), and more recently condensed representations of frequent itemsets have been studied [10,11,12]. The paper is organized as follows.…”

A Bayesian Approach for Classification Rule Mining in Quantitative Databases

“…For each mined rule π, a new Boolean attribute (feature) is created: the value of this new feature for a training object t of the data set r is (1) true if t supports the body of π, (0) false otherwise. This feature construction process is certainly the most straightforward but has also shown good predictive performance in several studies [8,14]. To provide predictions for new incoming (test) objects, we use a Selective Naive Bayes classifier (snb) on the recoded data set.…”