A Concise Review of Transfer Learning

Abstract

The availability of abundant labeled data in recent years led the researchers to introduce a methodology called transfer learning, which utilizes existing data in situations where there are difficulties in collecting new annotated data. Transfer learning aims to boost the performance of a target learner by applying another related source data. In contrast to the traditional machine learning and data mining techniques, which assume that the training and testing data lie from the same feature space and distribution, transfer learning can handle situations where there is a discrepancy between domains and distributions. These characteristics give the model the potential to utilize the available related source data and extend the underlying knowledge to the target task achieving better performance. This survey paper aims to give a concise review of traditional and current transfer learning settings, existing challenges, and related approaches.

Semi-Supervised Learning Approach for Fine Grained Human Hand Action Recognition in Industrial Assembly

“…For this purpose, a base model is first trained on labeled data, and this knowledge is then used as base initialization knowledge for new classes. This is done by replacing and re-training the final classification layers trough fine-tuning [Far+21]. In order to remain appropriate to the real condition in the production, only a small portion of marked data is initially used.…”

Exploring zero-shot models for cross-lingual and cross-domain transfer learning

“…Transfer learning consists of training a model on a specific source task and then using the learned weights to start the fine-tuning process on a second task of interest, which is commonly called a downstream task. It is an effective learning technique to reduce the amount of data and training requirements needed to achieve high performance on multiple tasks using neural networks (FARAHANI et al, 2021). This is especially useful when we do not have enough labeled data to fine-tune a good model, because transfer learning allows to get around this problem and develop skillful models that would be unfeasible in the absence of transfer learning.…”