Supervised Classification: Quite a Brief Overview

Domain adaptation has become a prominent problem setting in machine learning and related fields. This review asks the question: how can a classifier learn from a source domain and generalize to a target domain? We present a categorization of approaches, divided into, what we refer to as, sample-based, feature-based and inference-based methods. Sample-based methods focus on weighting individual observations during training based on their importance to the target domain. Feature-based methods revolve around on mapping, projecting and representing features such that a source classifier performs well on the target domain and inference-based methods incorporate adaptation into the parameter estimation procedure, for instance through constraints on the optimization procedure. Additionally, we review a number of conditions that allow for formulating bounds on the cross-domain generalization error. Our categorization highlights recurring ideas and raises questions important to further research.

show abstract

“…The reader is assumed to be familiar with supervised learning and risk minimization. For extensive overviews of these topics, see [71], [72].…”

Section: Domain Adaptationmentioning

confidence: 99%

A Review of Domain Adaptation without Target Labels

Kouw

Loog

2021

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

415

213

View full text Add to dashboard Cite

show abstract

“…This section is a brief overview of classification and risk minimization. For broader overviews, see [70,141]. Readers familiar with this material may skip to Section 3.…”

Section: Classificationmentioning

confidence: 99%

An introduction to domain adaptation and transfer learning

Kouw,

Loog

2018

Preprint

Self Cite

View full text Add to dashboard Cite

In machine learning, if the training data is an unbiased sample of an underlying distribution, then the learned classification function will make accurate predictions for new samples. However, if the training data is not an unbiased sample, then there will be differences between how the training data is distributed and how the test data is distributed. Standard classifiers cannot cope with changes in data distributions between training and test phases, and will not perform well. Domain adaptation and transfer learning are sub-fields within machine learning that are concerned with accounting for these types of changes. Here, we present an introduction to these fields, guided by the question: when and how can a classifier generalize from a source to a target domain? We will start with a brief introduction into risk minimization, and how transfer learning and domain adaptation expand upon this framework. Following that, we discuss three special cases of data set shift, namely prior, covariate and concept shift. For more complex domain shifts, there are a wide variety of approaches. These are categorized into: importance-weighting, subspace mapping, domain-invariant spaces, feature augmentation, minimax estimators and robust algorithms. A number of points will arise, which we will discuss in the last section. We conclude with the remark that many open questions will have to be addressed before transfer learners and domain-adaptive classifiers become practical.

show abstract

“…We use classification algorithms to learn the relationship between a set of features and the target class. 9 In our case we have heart-disease risk features such as age, cholesterol and the results of other medical tests and our class is the presence of heart disease for that patient.…”

Section: Introductionmentioning

confidence: 99%

Exploring feature selection and classification methods for predicting heart disease

et al. 2020

View full text Add to dashboard Cite

Machine learning has been used successfully to improve the accuracy of computer-aided diagnosis systems. This paper experimentally assesses the performance of models derived by machine learning techniques by using relevant features chosen by various feature-selection methods. Four commonly used heart disease datasets have been evaluated using principal component analysis, Chi squared testing, ReliefF and symmetrical uncertainty to create distinctive feature sets. Then, a variety of classification algorithms have been used to create models that are then compared to seek the optimal features combinations, to improve the correct prediction of heart conditions. We found the benefits of using feature selection vary depending on the machine learning technique used for the heart datasets we consider. However, the best model we created used a combination of Chi-squared feature selection with the BayesNet algorithm and achieved an accuracy of 85.00% on the considered datasets.

show abstract

Supervised Classification: Quite a Brief Overview

Cited by 16 publications

References 97 publications

A Review of Domain Adaptation without Target Labels

A Review of Domain Adaptation without Target Labels

An introduction to domain adaptation and transfer learning

Exploring feature selection and classification methods for predicting heart disease

Contact Info

Product

Resources

About