A Survey on Deep Transfer Learning

Abstract

As a new classification platform, deep learning has recently received increasing attention from researchers and has been successfully applied to many domains. In some domains, like bioinformatics and robotics, it is very difficult to construct a large-scale well-annotated dataset due to the expense of data acquisition and costly annotation, which limits its development. Transfer learning relaxes the hypothesis that the training data must be independent and identically distributed (i.i.d.) with the test data, which motivates us to use transfer learning to solve the problem of insufficient training data. This survey focuses on reviewing the current researches of transfer learning by using deep neural network and its applications. We defined deep transfer learning, category and review the recent research works based on the techniques used in deep transfer learning.

AIDeveloper: deep learning image classification in life science and beyond

“…In addition, custom-built CNNs as well as pre-trained models are supported. Pre-trained models can be used either for classifying new sets of images or for re-purposed training on a different classification task, a technique termed transfer learning (Tan et al, 2018). At the initiation of a training process, AID automatically generates the neural net architecture according to the requested input and output dimensions.…”

“…This issue can be tackled by increasing the amount of data. Here, due to the limited availability of labelled images, we applied a transfer learning approach, which was shown to reduce the need of data (Tan et al, 2018). We reached a validation accuracy of 92.1% by first training the model on CIFAR-10, and continuing training using the image data from differentiated MSCs.…”

“…For training, the user can either choose NNs of different complexities or use custom-built NNs. In addition, the interface allows the use of pre-trained models to transfer the learning process, i.e., when insufficient training data is available (Tan et al, 2018) or to shorten the training step.…”

High-throughput prediction of MHC Class I and Class II neoantigens with MHCnuggets

“…LSTM architectures excel at handling variable length sequence inputs and can learn longterm dependencies between non-contiguous elements, enabling an input encoding that does not require peptide shortening or splitting ( Figure 1B). The networks were trained with a transfer-learning protocol (32), which allows networks performing predictions for less wellcharacterized alleles to leverage information from extensively studied alleles ( Figure 1C). Transfer learning was also used to train networks using both binding affinity and HLAp datasets.…”

“…These values cover the vast majority lengths observed in naturally presented MHC bound peptides (12). The networks were trained with transfer learning (26), which allows networks for less well-characterized alleles to leverage information from extensively studied alleles ( Figure 1C). Transfer learning was also used to train networks combining binding affinity and HLAp datasets.…”