A robust semi-supervised classification method for transfer learning

Abstract

The transfer learning problem of designing good classifiers with a high generalization ability by using labeled samples whose distribution is different from that of test samples is an important and challenging research issue in the fields of machine learning and data mining. This paper focuses on designing a semi-supervised classifier trained by using unlabeled samples drawn by the same distribution as test samples, and presents a semi-supervised classification method to deal with the transfer learning problem, based on a hybrid discriminative and generative model. Although JESS-CM is one of the most successful semi-supervised classifier design frameworks and has achieved the best published results in NLP tasks, it has an overfitting problem in transfer learning settings that we consider in this paper. We expect the overfitting problem to be mitigated with the proposed method, which utilizes both labeled and unlabeled samples for the discriminative training of classifiers. We also present a refined objective that formalizes the training algorithm and classifier form. Our experimental results for text classification using three typical benchmark test collections confirmed that the proposed method outperformed the JESS-CM framework with most transfer learning settings.

Modeling and classification of service behaviors

“…Two strategies are used for services classification: supervised and unsupervised. For a supervised classification, such as the ones suggested in Fujino et al, 2010;He et al, 2004;Oldham et al, 2005;Crasso et al, 2008), two phases are required: training and classification phases. The training phase is needed to build a classifier from a collection of categorized services, while the classification phase associates a new service to one or more classes.…”

Word Sense Disambiguation by Combining Labeled Data Expansion and Semi-Supervised Learning Method

“…, s K } given the feature vector x of a word instance is modeled by a combination of discriminative and generative models, P d (s|x; W) and p g (x, s; ), where W and are the respective parameters of these models. By applying the classifier form and training method presented in Fujino et al [2010], we define P(s|x) as the following. We also provide objective function J for the parameter estimation of P(s k |x; W, , β) by using labeled and unlabeled datasets,…”

“…The local optimal solution of W and around an initial value can be obtained by an iterative process such as the EM algorithm [Dempster et al 1977]. Namely, the MHLE-based semi-supervised WSD classifier is constructed by combining the discriminative and generative models trained on both labeled and unlabeled samples (See Fujino et al [2010] for details of the combination and training methods).…”

“…We employed a maximum entropy model (multinomial logistic regression model) and a naive Bayes model as P d (s|x; W) and p g (x, s; ), as well as the text classifier presented in Fujino et al [2010]. In the naive Bayes model, the probability distribution of x = (x 1 , .…”

Multi-Source Tri-Training Transfer Learning

“…As the machine learning theory continues to develop, it is impossible to meet the needs of classifier training merely by relying on a limited number of labeled samples and it becomes extremely important to make use of unlabeled samples for co-training. Fujino et al [6] designed a semi-supervised classifier (MHLE) trained by using unlabeled samples drawn by the same distribution as testing samples, and presented a semi-supervised classification method to deal with the transfer learning problem. The MHLE needs to designate a discriminative and generative model for a specific problem beforehand, which requires much domain knowledge.…”