Inter- and Intra-domain Knowledge Transfer for Related Tasks in Deep Character Recognition

Abstract

Pre-training a deep neural network on the ImageNet dataset is a common practice for training deep learning models, and generally yields improved performance and faster training times. The technique of pre-training on one task and then retraining on a new one is called transfer learning. In this paper we analyse the effectiveness of using deep transfer learning for character recognition tasks. We perform three sets of experiments with varying levels of similarity between source and target tasks to investigate the behaviour of different types of knowledge transfer. We transfer both parameters and features and analyse their behaviour. Our results demonstrate that no significant advantage is gained by using a transfer learning approach over a traditional machine learning approach for our character recognition tasks. This suggests that using transfer learning does not necessarily presuppose a better performing model in all cases.

Investigating Transfer Learning in Graph Neural Networks

“…One reason for the success of deep learning models is their ability to transfer previous learning to new tasks. In image classification, this transfer leads to more robust models and faster training [8][9][10][11][12][13]. Despite the importance of transfer in deep learning, there has been little insight into the nature of transferring relational knowledge-that is, the representations learnt by graph neural networks.…”

Improving Cause-of-Death Classification from Verbal Autopsy Reports

“…Transfer learning [13,14,15,16,17] allows for knowledge derived from tasks rich in data to be applied to tasks, languages, or domains where data is limited. It consist of two steps, pretraining on one task or domain (source) and domain adaptation where the learned representations are used in a different task, domain or language (target).…”