Pre-trained Word Embedding based Parallel Text Augmentation Technique for Low-Resource NMT in Favor of Morphologically Rich Languages

Hailu, Tulu Tilahun; Yu, Jiguo; Fantaye, Tessfu Geteye

doi:10.1145/3331453.3361309

Cited by 2 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on the text augmentation techniques proposed in [14], we generated three groups of paraphrases, which can be referred as paraphrase generation system 1, 2 and 3 (PG1, PG2, and PG3). The first system (PG1) is intended to generate paraphrases by replacing words of the original text with the first most similar word in the vector space whereas the second and the third systems are intended to generate paraphrases by replacing the words in the original text with the second and third most similar words in the vector space respectively.…”

Section: Paraphrase Generation Methodsmentioning

confidence: 99%

“…Similarly, automatically generated paraphrases have been used to improve various NLP models, for example, question-answering [5] [6]; information extraction and retrieval [7] [8]; relation extraction [9]; text summarization [10] [11]; machine translation [12] [13] [14]; automatic generation of reference translation [15] [16], etc. As a result, recently, in the field of NLP, automatic paraphrase generation received increasing attention.…”

Section: Introductionmentioning

confidence: 99%

“…• Based on lesson learned from study in [14], we generate paraphrases and prepare them for analyzing intrinsic and extrinsic evaluation strategies and assess their relationships.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Intrinsic and Extrinsic Automatic Evaluation Strategies for Paraphrase Generation Systems

Hailu¹,

Yu²,

Fantaye³

2020

JCC

Self Cite

View full text Add to dashboard Cite

Paraphrase is an expression of a text with alternative words and orders to achieve a better clarity. Paraphrases have been found vital for augmenting training dataset, which aid to enhance performance of machine learning models that intended for various natural language processing (NLP) tasks. Thus, recently, automatic paraphrase generation has received increasing attention. However, evaluating quality of generated paraphrases is technically challenging. In the literature, the importance of generated paraphrases is tended to be determined by their impact on the performance of other NLP tasks. This kind of evaluation is referred as extrinsic evaluation, which requires high computational resources to train and test the models. So far, very little attention has been paid to the role of intrinsic evaluation in which quality of generated paraphrase is judged against predefined ground truth (reference paraphrases). In fact, it is also very challenging to find ideal and complete reference paraphrases. Therefore, in this study, we propose semantic or meaning oriented automatic evaluation metric that helps to evaluate quality of generated paraphrases against the original text, which is an intrinsic evaluation approach. Further, we evaluate quality of the paraphrases by assessing their impact on other NLP tasks, which is an extrinsic evaluation method. The goal is to explore the relationship between intrinsic and extrinsic evaluation methods. To ensure the effectiveness of proposed evaluation methods, extensive experiments are done on different publicly available datasets. The experimental results demonstrate that our proposed intrinsic and extrinsic evaluation strategies are promising. The results further reveal that there is a significant correlation between intrinsic and extrinsic evaluation approaches.

show abstract

Section: Paraphrase Generation Methodsmentioning

confidence: 99%