Meta-data: Characterization of Input Features for Meta-learning

Castiello, Ciro; Castellano, Giovanna; Fanelli, Anna Maria

doi:10.1007/11526018_45

Cited by 67 publications

(63 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Sohn (1999) notices that some of the characteristics are highly correlated, and she omits the redundant ones in her study. Castiello et al (2005) provide formulas for different data characteristics (meta-features) and theoretically discuss their relevance. However, their assumptions are based on intuition and theoretical knowledge.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

On the predictive power of meta-features in OpenML

Bilalli

Abelló

Aluja-Banet

2017

International Journal of Applied Mathematics and Computer Science

View full text Add to dashboard Cite

The demand for performing data analysis is steadily rising. As a consequence, people of different profiles (i.e., nonexperienced users) have started to analyze their data. However, this is challenging for them. A key step that poses difficulties and determines the success of the analysis is data mining (model/algorithm selection problem). Meta-learning is a technique used for assisting non-expert users in this step. The effectiveness of meta-learning is, however, largely dependent on the description/characterization of datasets (i.e., meta-features used for meta-learning). There is a need for improving the effectiveness of meta-learning by identifying and designing more predictive meta-features. In this work, we use a method from exploratory factor analysis to study the predictive power of different meta-features collected in OpenML, which is a collaborative machine learning platform that is designed to store and organize meta-data about datasets, data mining algorithms, models and their evaluations. We first use the method to extract latent features, which are abstract concepts that group together meta-features with common characteristics. Then, we study and visualize the relationship of the latent features with three different performance measures of four classification algorithms on hundreds of datasets available in OpenML, and we select the latent features with the highest predictive power. Finally, we use the selected latent features to perform meta-learning and we show that our method improves the meta-learning process. Furthermore, we design an easy to use application for retrieving different meta-data from OpenML as the biggest source of data in this domain.

show abstract

Section: Related Workmentioning

confidence: 99%

“…There have been many studies with regard to the use (what kinds of meta-data to be used) (Castiello et al, 2005) and selection (which are the most relevant) (Kalousis and Hilario, 2001) of meta-features or meta-data (Bilalli et al, 2016) in general. However, these studies have been performed independently and in specific domains.…”

Section: Introductionmentioning

confidence: 99%

On the predictive power of meta-features in OpenML

Bilalli

Abelló

Aluja-Banet

2017

International Journal of Applied Mathematics and Computer Science

View full text Add to dashboard Cite

show abstract

“…The basic examples of meta-learning can be found in the variable bias management system (VBMS) [47] which automatically choses a learning algorithm based on two meta-parameters: the number of features and the number of vectors, often extended by descriptors representing the number of features of given type. An extension of this idea is a meta-learning system which is based on a set of meta-variables describing aggregated statistical and information theory properties of individual attributes and class labels [5,6,[48][49][50]. Commonly reported meta-features of this type include: canonical correlation for the best single combination of features (called cancor1), canonical correlation for the best single combination of features orthogonal to cancor1, the first normalized eigenvalue of the canonical discriminant matrix, the second normalized eigenvalue of the canonical discriminant matrix, the mean kurtosis of attributes of T, the mean skewness of attributes of T, the mean mutual information of class and feature, joint entropy of a class variable and attribute, and entropy of classes, etc.…”

Section: Meta-learningmentioning

confidence: 99%

“…It is based on representing the training set, which consists of n feature vectors by a single meta-date instance, and then uses it as an input to the meta-model which returns the estimate of the accuracy without training the actual (base) model [6].…”

Section: Introductionmentioning

confidence: 99%

Instance Selection for Classifier Performance Estimation in Meta Learning

Blachnik

2017

Entropy

View full text Add to dashboard Cite

Abstract:Building an accurate prediction model is challenging and requires appropriate model selection. This process is very time consuming but can be accelerated with meta-learning-automatic model recommendation by estimating the performances of given prediction models without training them. Meta-learning utilizes metadata extracted from the dataset to effectively estimate the accuracy of the model in question. To achieve that goal, metadata descriptors must be gathered efficiently and must be informative to allow the precise estimation of prediction accuracy. In this paper, a new type of metadata descriptors is analyzed. These descriptors are based on the compression level obtained from the instance selection methods at the data-preprocessing stage. To verify their suitability, two types of experiments on real-world datasets have been conducted. In the first one, 11 instance selection methods were examined in order to validate the compression-accuracy relation for three classifiers: k-nearest neighbors (kNN), support vector machine (SVM), and random forest. From this analysis, two methods are recommended (instance-based learning type 2 (IB2), and edited nearest neighbor (ENN)) which are then compared with the state-of-the-art metaset descriptors. The obtained results confirm that the two suggested compression-based meta-features help to predict accuracy of the base model much more accurately than the state-of-the-art solution.

show abstract

“…Meta-features have been commonly employed in several works [29,30,31,32] and projects, such as StatLog and METAL. Table 2 reports a selection of meta-features; for a thorough description of those measures the interested reader is addressed to [33].…”

Section: About the Meta-feature Extractionmentioning

confidence: 99%

Computational Intelligence for Meta-Learning: A Promising Avenue of Research

Castiello

Fanelli

2011

Studies in Computational Intelligence

Self Cite

View full text Add to dashboard Cite

Abstract. The common practices of machine learning appear to be frustrated by a number of theoretical results denying the possibility of any meaningful implementation of a "superior" learning algorithm. However, there exist some general assumptions that, even when overlooked, preside the activity of researchers and practitioners. A thorough reflection over such essential premises brings forward the meta-learning approach as the most suitable for escaping the long-dated riddle of induction claiming also an epistemologic soundness. Several examples of meta-learning models can be found in literature, yet the combination of computational intelligence techniques with meta-learning models still remains scarcely explored. Our contribution to this particular research line consists in the realisation of Mindful, a meta-learning system based on the neurofuzzy hybridisation. We present the Mindful system firstly situating it inside the general context of the meta-learning frameworks proposed in literature. Finally, a complete session of experiments is illustrated, comprising both base-level and meta-level learning activity. The appreciable experimental results underline the suitability of the Mindful system for managing past accumulated learning experience while facing novel tasks.

show abstract

Meta-data: Characterization of Input Features for Meta-learning

Cited by 67 publications

References 6 publications

On the predictive power of meta-features in OpenML

On the predictive power of meta-features in OpenML

Instance Selection for Classifier Performance Estimation in Meta Learning

Computational Intelligence for Meta-Learning: A Promising Avenue of Research

Contact Info

Product

Resources

About