Bagging based ensemble transfer learning

Abstract

Nowadays, transfer learning is one of the main research areas in machine learning that is helpful for labeling the data with low cost. In this paper, we propose a novel bagging-based ensemble transfer learning (BETL). The BETL framework includes three operations: Initiate, Update, and Integrate. In the Initiate operation, we use bootstrap sampling to divide the source data into many subsets, and add the labeled data from the target domain into these subsets separately so that the source data and the target data arrive at a reasonable ratio, then we learn as many initial classifiers as the elements of an ensemble. In the Update operation, we utilize the initial classifiers and an updateable classifier to repeatedly label the data that hasn't been labeled yet in the target domain, and then, add the newly labeled data into the target domain to renew the updateable classifier. In the Integrate operation, we integrate the updated classifiers from each iteration into a pool to predict the labels of the test data via the majority vote strategy. In order to demonstrate the effectiveness of our method in the classification process, we conduct experiments on UCI data set, real world data set, and text data set. The results show that our method can effectively label the unlabeled data in the target domain, which greatly enhances the performance of target domain.

A Novel Relational-Based Transductive Transfer Learning Method for PolSAR Images via Time-Series Clustering

“…1The transfer precision is high for objects which are easy to classify, but when the object is difficult to distinguish, its transfer precision will be very unreliable. 2The weak classifier used in the corresponding literature [2,8] was the NB classifier, but in this study, the effect of this classifier was found to be not ideal. After comparing decision tree, NB, and SVM classifiers, it was found that using SVM as the weak classifier could allow TrBagg and BETL to reach a higher precision.…”

“…The sample usage in each experiment is shown in Table 1 below: Although TrBagg and BETL only need at least one labeled sample in the target domain, the stability of these two methods is poor when the target domain labeled samples are too few, so we expanded them to five in this experiment. For the selection of the weak classifier, although both TrBagg and BETL use a naive Bayes (NB) classifier as the weak classifier in the corresponding references [2,8], in order to achieve a higher precision for these two methods, after comparing various weak classifiers, we used a support vector machine (SVM) classifier as the weak classifier for TrBagg and BETL.…”

“…Firstly, with regard to "what to transfer", transfer learning can be categorized into four approaches: instance-based transfer, feature-representation-based transfer, parameter-based transfer, and relational knowledge transfer. Taking remote sensing image classification as an example, instance-based transfer methods, such as the methods proposed in [2][3][4][5][6][7][8], use partial training samples in the source domain to improve the performance of the model of the target domain in the model training. The feature-representation-based transfer methods, such as the methods proposed in [9][10][11][12], learn a more effective feature expression from the source domain, so that the target domain classifier built in the new feature space has a better performance.…”

“…In order to evaluate the effectiveness of the proposed method, two other transfer learning algorithms were introduced to carry out comparative experiments: transfer bagging (TrBagg) [2] and bagging-based ensemble transfer learning (BETL) [8].…”

Application of BP Neural Network Ensemble Model Based on Bagging Algorithm

“…BP neural network is one of the most widely used neural network models, which can classify prediction problems and possess highly nonlinear mapping and accurate function fitting, as well as the ability of self-organizing, self-learning, self-adaptive and parallel processing [21]. Bagging algorithm is an ensemble learning algorithm, which is an ensemble method to improve the accuracy of learning algorithms by sampling samples and feature attributes to generate multiple independent models [22]. The problem in this paper can be formally described as follows: Given coal and gas outburst data set D containing m data records…”