A scalable robust and automatic propositionalization approach for Bayesian classification of large mixed numerical and categorical data

Abstract

Companies want to extract value from their relational databases. This is the aim of relational data mining. Propositionalization is one possible approach to relational data mining. Propositionalization adds new attributes, called features, to the main table, leading to an attribute-value representation, a single table, on which a propositional learner can be applied. However, current relational databases are large and composed of mixed, numerical and categorical, data. Moreover, the specificity of relational data is to involve one-to-many relationships. As an example of such data, consider customers purchasing products: each customer can purchase several products. Therefore, there is a need for techniques able to learn complex aggregates. Learning such features means to explore a combinatorial, possibly infinite, space and such an approach is prone to overfitting. We introduce a propositionalization approach dedicated to a robust Bayesian classifier. It efficiently samples a given number of features in the language bias, following a distribution over the complex aggregates. This distribution is also used to penalize complex aggregates in the regularization of the robust Bayesian classifier. Experiments show that it performs better than state-of-the-art methods on most investigated benchmarks and can deal with large datasets more easily. A new real, large, mixed relational dataset is introduced which confirms the ability of our approach to learn complex aggregates.

LazyBum: Decision Tree Learning Using Lazy Propositionalization

“…As a result, our OneBM and LazyBum versions are able to generate the same features. -MODL [3,4] is a recent static propositionalization approach included in the Khiops data mining tool. -nFOIL [16].…”

“…Many propositionalization approaches have been proposed in the past and succesfully applied to various domains, such as information extraction and word sense disambiguation [10,14,4,23,24]. To better position LazyBum with respect to the state of the art, we categorize these approaches along four dimensions.…”

“…The ILP learners Tilde and FORF included in the ACE system [25] allow for complex aggregates to be used. A recent propositionalization approach that considers complex aggregates is MODL [3,4], which is included in the Khiops data mining tool. MODL was designed to deal with a possibly infinite feature space.…”

Interpretable Feature Construction for Time Series Extrinsic Regression

“…To extract features such as in the illustrative example, our approach is based on (i) the computation of multiple yet simple representations of time series, and their storage in a relational data scheme, (ii) a recent approach for feature engineering through propositionalisation [6] and its extension for regression problems. In the following, we describe these two steps in order to make the paper self-contained.…”

“…In this paper, we exploit a relational machine learning approach [6] for interpretable feature construction and selection and suggest an extension for TSER problems. As a motivating example, we consider the AppliancesEnergy data [20].…”