A Joint Back-Translation and Transfer Learning Method for Low-Resource Neural Machine Translation

Luo, Gong Xu; Yang, Ya; Dong, Rui; Chen, Yan Hong; Zhang, Wen Bo

doi:10.1155/2020/6140153

Cited by 7 publications

(5 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The goal of the fine-tuning phase is to further optimize the performance of the translation task for low-resource languages on the basis of the pre-trained model. Specifically, the selected pre-trained model is loaded with a bilingual parallel dataset of the target language for fine-tuning: (3) Hierarchical fine-tuning: first fine-tuning within a large family of languages, and then targeted finetuning to target low-resource languages, which helps to gradually focus on more specific and scarce language features [22].…”

Section: Fine-tuning Strategies and Low-resource Translation Performa...mentioning

confidence: 99%

Migration Learning and Multi-View Training for Low-Resource Machine Translation

Yan,

Lin,

Zhao

2024

ijacsa

View full text Add to dashboard Cite

This paper discusses the main challenges and solution strategies of low-resource machine translation, and proposes a novel translation method combining migration learning and multi-view training. In a low-resource environment, neural machine translation models are prone to problems such as insufficient generalization performance, inaccurate translation of long sentences, difficulty in processing unregistered words, and inaccurate translation of domain-specific terms due to their heavy reliance on massively parallel corpora. Migration learning gradually adapts to the translation tasks of low-resource languages in the process of fine-tuning by borrowing the general translation knowledge of high-resource languages and utilizing pre-training models such as BERT, XLM-R, and so on. Multiperspective training, on the other hand, emphasizes the integration of source and target language features from multiple levels, such as word level, syntax and semantics, in order to enhance the model's comprehension and translation ability under limited data conditions. In the experiments, the study designed an experimental scheme containing pre-training model selection, multi-perspective feature construction, and migration learning and multi-perspective fusion, and compared the performance with randomly initialized Transformer model, pre-training-only model, and traditional statistical machine translation model. The experiments demonstrate that the model with multi-view training strategy significantly outperforms the baseline model in evaluation metrics such as BLEU, TER, and ChrF, and exhibits stronger robustness and accuracy in processing complex language structures and domain-specific terminology.

show abstract

Section: Fine-tuning Strategies and Low-resource Translation Performa...mentioning

confidence: 99%

Migration Learning and Multi-View Training for Low-Resource Machine Translation

Yan,

Lin,

Zhao

2024

ijacsa

View full text Add to dashboard Cite

show abstract

“…NMT has the challenge of handling rare words. Researchers have also tried to address this problem and achieved a satisfactory result as well [16].…”

Section: ( )mentioning

confidence: 99%

“…These searching strategies with hyper-parameters to train a model to achieve maximum accuracy with its new set of trained parameters are called hyper-parameter optimization techniques. However, in NMT, apart from hyper-parameter optimization, there are other approaches such as linguistic features, representation of abstract meaning on semantic graphs, and various other novel approaches used by researchers to improve the performance of NMT systems, specifically when using low-resource language pairs [15][16][17][18]. The aims of this paper are to:…”

Section: Introductionmentioning

confidence: 99%

Hyper-parameter optimization in neural-based translation systems: A case study

Datta,

Joshi,

Gupta

2023

International Journal on Smart Sensing and Intelligent Systems

View full text Add to dashboard Cite

Machine translation (MT) is an important use case in natural language processing (NLP) that converts a source language to a target language automatically. Modern intelligent system or artificial intelligence (AI) uses a machine learning approach and the machine has acquired learning ability using datasets. Nowadays, in the MT domain, the neural machine translation (NMT) system has almost replaced the statistical machine translation (SMT) system. The NMT systems use a deep learning framework in their implementation. To achieve higher accuracy during the training of the NMT model, extensive hyper-parameter tuning is required. The paper highlights the significance of hyper-parameter tuning in various machine learning algorithms. And as a case study, in-house experimentation was conducted on a low-resource English–Bangla language pair by designing an NMT system and the significance of various hyper-parameter optimizations was analyzed while evaluating its performance with an automatic metric BLEU. The BLEU scores obtained for the first, second, and third randomly picked test sentences are 4.1, 3.2, and 3.01, respectively.

show abstract

“…Finding and fine tuning the network model of classifiers suitable for specific data sets is one of these methods, such as [ 25 ] describing the proposed approach for text classification in an unbalanced data environment based on an implementing individual LSTM neural network. Data augmentation is also a frequently used way in scenarios with few samples or unbalanced categories [ 26 – 29 ], such as back translation [ 26 ], which has become an effective way of data augmentation; for example, researchers translate some English text into Chinese and then translate them back into English, they can get various new training data, and the size of the dataset is doubled. Oversampling [ 27 ] and undersampling [ 28 ] are both common methods to deal with unbalanced datasets.…”

Section: Introductionmentioning

confidence: 99%

A Novel Approach of Feature Space Reconstruction with Three-Way Decisions for Long-Tailed Text Classification

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Text classification is widely studied by researchers in the natural language processing field. However, real-world text data often follow a long-tailed distribution as the frequency of each class is typically different. The performance of current mainstream learning algorithms in text classification suffers when the training data are highly imbalanced. The problem can get worse when the categories with fewer data are severely undersampled to the extent that the variation within each category is not fully captured by the given data. At present, there are a few studies on long-tailed text classification which put forward effective solutions. Encouraged by the progress of handling long-tailed data in the field of image, we try to integrate effective ideas into the field of long-tailed text classification and prove the effectiveness. In this paper, we come up with a novel approach of feature space reconstruction with the help of three-way decisions (3WDs) for long-tailed text classification. In detail, we verify the rationality of using a 3WD model for feature selection in long-tailed text data classification, propose a new feature space reconstruction method for long-tailed text data for the first time, and demonstrate how to effectively generate new samples for tail classes in reconstructed feature space. By adding new samples, we enrich the representing information of tail classes, to improve the classification results of long-tailed text classification. After some comparative experiments, we have verified that our model is an effective strategy to improve the performance of long-tailed text classification.

show abstract

A Joint Back-Translation and Transfer Learning Method for Low-Resource Neural Machine Translation

Cited by 7 publications

References 23 publications

Migration Learning and Multi-View Training for Low-Resource Machine Translation

Migration Learning and Multi-View Training for Low-Resource Machine Translation

Hyper-parameter optimization in neural-based translation systems: A case study

A Novel Approach of Feature Space Reconstruction with Three-Way Decisions for Long-Tailed Text Classification

Contact Info

Product

Resources

About