Autoencoder based sample selection for self-taught learning

Abstract

Self-taught learning is a technique that uses a large number of unlabeled data as source samples to improve the task performance on target samples. Compared with other transfer learning techniques, self-taught learning can be applied to a broader set of scenarios due to the loose restrictions on source data. However, knowledge transferred from source samples that are not sufficiently related to the target domain may negatively influence the target learner, which is referred to as negative transfer. In this paper, we propose a metric for the relevance between a source sample and target samples. To be more specific, both source and target samples are reconstructed through a single-layer autoencoder with a linear relationship between source samples and target samples simultaneously enforced. An 2,1 -norm sparsity constraint is imposed on the transformation matrix to identify source samples relevant to the target domain. Source domain samples that are deemed relevant are assigned pseudo-labels reflecting their relevance to target domain samples, and are combined with target samples in order to provide an expanded training set for classifier training. Local data structures are also preserved during source sample selection through spectral graph analysis. Promising results in extensive experiments show the advantages of the proposed approach.

A Comparative Assessment of Machine-Learning Techniques for Forest Degradation Caused by Selective Logging in an Amazon Region Using Multitemporal X-Band SAR Images

“…Convolutional neural network (CNN) tests are also suggested, which have great potential for generalization in areas not seen before, as they use contextual information and are not strongly affected by absolute pixel values [85]. Other suggested approaches to reduce the overfitting problem are endless learning [86] and self-taught learning [87], which can be tested in the future.…”

Prior-Guided Transfer Learning for Enhancing Item Representation in E-commerce

“…Unsupervised domain adaptation assumes that only unlabeled data are available in the target domain. The core idea is to find a shared feature space which can reduce the domain distribution divergence (Lee et al 2007;Qiu et al 2017;Wang and Mahadevan 2008;Jiang et al 2019;Feng, Yu, and Duarte 2020). Pan et al (Pan et al 2009) and Long et al (Long et al 2015) proposed to minimize the Maximum Mean Discrepancies (MMD) to align the domain distributions.…”

Artificial neural network technique for improving prediction of credit card default: A stacked sparse autoencoder approach

“…The autoencoder was also optimized using gradient-based learning. Lastly, sparse autoencoder networks have achieved remarkable performance in representation learning [21], [22]. However, better representation learning can be gotten when multiple sparse autoencoders are stacked and optimized effectively, which is the focus of this research.…”