Improving SVM accuracy by training on auxiliary data sources

Wu, Pengcheng; Dietterich, Thomas G.

doi:10.1145/1015330.1015436

Cited by 229 publications

(136 citation statements)

References 8 publications

Supporting

Mentioning

135

Contrasting

Unclassified

Order By: Relevance

“…This asymmetric case, or transfer learning, requires the assumption of an asymmetric dependency structure between tasks. Existing approaches include reweighting-based methods [16,3,4] or learning of shared feature spaces. An alternative has been to, in effect, use a symmetric multi-task learning method in an asymmetric mode, by using the model learned from auxiliary tasks as a prior for the target task [9,12,17].…”

Section: Symmetric and Asymmetric Multi-task Learningmentioning

confidence: 99%

Focused Multi-task Learning Using Gaussian Processes

Leen

Peltonen

Kaski

2011

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

Abstract. Given a learning task for a data set, learning it together with related tasks (data sets) can improve performance. Gaussian process models have been applied to such multi-task learning scenarios, based on joint priors for functions underlying the tasks. In previous Gaussian process approaches, all tasks have been assumed to be of equal importance, whereas in transfer learning the goal is asymmetric: to enhance performance on a target task given all other tasks. In both settings, transfer learning and joint modelling, negative transfer is a key problem: performance may actually decrease if the tasks are not related closely enough. In this paper, we propose a Gaussian process model for the asymmetric setting, which learns to "explain away" non-related variation in the additional tasks, in order to focus on improving performance on the target task. In experiments, our model improves performance compared to single-task learning, symmetric multi-task learning using hierarchical Dirichlet processes, and transfer learning based on predictive structure learning.

show abstract

Section: Symmetric and Asymmetric Multi-task Learningmentioning

confidence: 99%

Focused Multi-task Learning Using Gaussian Processes

Leen

Peltonen

Kaski

2011

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

show abstract

“…Existing approaches include reweighting-based methods (Wu and Dietterich 2004;Bickel et al 2008Bickel et al , 2009 or learning of shared feature spaces. An alternative has been to, in effect, use a symmetric multi-task learning method in an asymmetric mode, by using the model learned from auxiliary tasks as a prior for the target task Raina et al 2005;Xue et al 2007).…”

Section: Symmetric and Asymmetric Multi-task Learningmentioning

confidence: 99%

Focused multi-task learning in a Gaussian process framework

2012

View full text Add to dashboard Cite

Multi-task learning, learning of a set of tasks together, can improve performance in the individual learning tasks. Gaussian process models have been applied to learning a set of tasks on different data sets, by constructing joint priors for functions underlying the tasks. In these previous Gaussian process models, the setting has been symmetric in the sense that all the tasks have been assumed to be equally important, whereas in settings such as transfer learning the goal is asymmetric, to enhance performance in a target task given the other tasks. We propose a focused Gaussian process model which introduces an "explaining away" model for each of the additional tasks to model their non-related variation, in order to focus the transfer to the task-of-interest. This focusing helps reduce the key problem of negative transfer, which may cause performance to even decrease if the tasks are not related closely enough. In experiments, our model improves performance compared to single-task learning, symmetric multi-task learning using hierarchical Dirichlet processes, transfer learning based on predictive structure learning, and symmetric multi-task learning with Gaussian processes.

show abstract

“…In image, text analysis or robotics many methods have been devised for knowledge transfer. Related machine learning subjects include: learning from hints [104], lifelong learning [105], multi-task learning [106], cross-domain learning [107,108], cross-category learning [109] and selftaught learning [110]. EigenTransfer algorithm [111] tries to unify various transfer learning ideas representing the target task by a graph.…”

Section: Transfer Of Knowledgementioning

confidence: 99%

Optimal Support Features for Meta-Learning

Duch

Maszczyk

Grochowski

2011

Studies in Computational Intelligence

View full text Add to dashboard Cite

Meta-learning has many aspects, but its final goal is to discover in an automatic way many interesting models for a given data. Our early attempts in this area involved heterogeneous learning systems combined with a complexity-guided search for optimal models, performed within the framework of (dis)similarity based methods to discover "knowledge granules". This approach, inspired by neurocognitive mechanisms of information processing in the brain, is generalized here to learning based on parallel chains of transformations that extract useful information granules and use it as additional features. Various types of transformations that generate hidden features are analyzed and methods to generate them are discussed. They include restricted random projections, optimization of these features using projection pursuit methods, similarity-based and general kernel-based features, conditionally defined features, features derived from partial successes of various learning algorithms, and using the whole learning models as new features. In the enhanced feature space the goal of learning is to create image of the input data that can be directly handled by relatively simple decision processes. The focus is on hierarchical methods for generation of information, starting from new support features that are discovered by different types of data models created on similar tasks and successively building more complex features on the enhanced feature spaces. Resulting algorithms facilitate deep learning, and also enable understanding of structures present in the data by visualization of the results of data transformations and by creating logical, fuzzy and prototype-based rules based on new features. Relations to various machine-learning approaches, comparison of results, and neurocognitive inspirations for meta-learning are discussed.

show abstract

Improving SVM accuracy by training on auxiliary data sources

Cited by 229 publications

References 8 publications

Focused Multi-task Learning Using Gaussian Processes

Focused Multi-task Learning Using Gaussian Processes

Focused multi-task learning in a Gaussian process framework

Optimal Support Features for Meta-Learning

Contact Info

Product

Resources

About