Domain Space Transfer Extreme Learning Machine for Domain Adaptation

Abstract

Extreme learning machine (ELM) has been applied in a wide range of classification and regression problems due to its high accuracy and efficiency. However, ELM can only deal with cases where training and testing data are from identical distribution, while in real world situations, this assumption is often violated. As a result, ELM performs poorly in domain adaptation problems, in which the training data (source domain) and testing data (target domain) are differently distributed but somehow related. In this paper, an ELM-based space learning algorithm, domain space transfer ELM (DST-ELM), is developed to deal with unsupervised domain adaptation problems. To be specific, through DST-ELM, the source and target data are reconstructed in a domain invariant space with target data labels unavailable. Two goals are achieved simultaneously. One is that, the target data are input into an ELM-based feature space learning network, and the output is supposed to approximate the input such that the target domain structural knowledge and the intrinsic discriminative information can be preserved as much as possible. The other one is that, the source data are projected into the same space as the target data and the distribution distance between the two domains is minimized in the space. This unsupervised feature transformation network is followed by an adaptive ELM classifier which is trained from the transferred labeled source samples, and is used for target data label prediction. Moreover, the ELMs in the proposed method, including both the space learning ELM and the classifier, require just a small number of hidden nodes, thus maintaining low computation complexity. Extensive experiments on real-world image and text datasets are conducted and verify that our approach outperforms several existing domain adaptation methods in terms of accuracy while maintaining high efficiency.

Extreme Learning Machine Based on Maximum Weighted Mean Discrepancy for Unsupervised Domain Adaptation

“…is the mean point of samples belong to class c in source or target domain. By incorporating (17) and (19), the joint MWMD optimization problem can be presented as follows:…”

“…The results are compared with several state-of-the-art related methods, including supervised classification methods, semisupervised and unsupervised domain adaptation methods. For Office datasets and hand-written digit datasets, two traditional classification algorithms and 5 domain adaptation approaches were compared, including SVM [35], ELM [12], TCA [36], JDA [31], DAELM (DAELM-S and DAELM-T) [15], CDELM-M [21] and DST-ELM [19]. For COIL20 dataset, we chose some other DA algorithms, such as PCA, GFK [34], SA [37] and JUC-SDELM [20].…”

“…To deal with the cases where there is no labeled data in target domain, Liu et al [18] have proposed a unified subspace transfer framework called cross-domain extreme learning machine (CdELM), in which the ELM was well-exploited to learn a shared subspace across domains. Furthermore, Chen et al [19] have developed another ELM-based space learning algorithm, domain space transfer ELM (DST-ELM), both source and target data were reconstructed in a domain invariant space, and the distribution distance of the two domains was also minimized here. Instead of learning a shared subspace by ELM, Zhang et al [20] have integrated the scalable factor into discriminative ELM (DELM) to strengthen the discriminative capacity of the ELM classifier, and proposed a joint unsupervised cross-domain model via scalable discriminative ELM (JUC-SDELM).…”

Hyperspectral Image Classification Based on Domain Adaptation Broad Learning

“…2) Transfer learning methods: TCA [44], JDA [18], DST-ELM [19], and manifold embedded distribution alignment (MEDA) [39]. 3) Deep domain adaptation method: DANN [20].…”

“…For example, Long et al [18] proposed a joint distribution adaptation (JDA) that can jointly adapt both the marginal and conditional distributions. Chen et al [19] proposed to reduce the domain distribution difference between the source and target domains using extreme learning machine framework, named domain space transfer ELM (DST-ELM). Ganin and Lempitsky [20] proposed a domain-adversarial neural network (DANN) to select transferable features from different domains by introducing the adversarial mechanism into deep transfer network.…”