Knowledge Distillation: A Method for Making Neural Machine Translation More Efficient

Jooste, Wandri; Haque, Rejwanul; Way, Andy

doi:10.3390/info13020088

Cited by 12 publications

(6 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…13 In light of such developments, a 'green report' was incorporated into adaptNMT whereby the kgCO 2 generated during model development is logged. This is very much in line with the industry trend of quantifying the impact of NLP on the environment; indeed, Jooste et al (2022a) have demonstrated that highperforming MT systems can be built with much lower footprints, which not only reduce emissions, but also in the post-deployment phase deliver savings of almost 50% in energy costs for a real translation company.…”

Section: Stochastic Nuancessupporting

confidence: 70%

adaptNMT: an open-source, language-agnostic development environment for neural machine translation

Lankford

Afli

Way

2023

Lang Resources & Evaluation

Self Cite

View full text Add to dashboard Cite

AbstractadaptNMT streamlines all processes involved in the development and deployment of RNN and Transformer neural translation models. As an open-source application, it is designed for both technical and non-technical users who work in the field of machine translation. Built upon the widely-adopted OpenNMT ecosystem, the application is particularly useful for new entrants to the field since the setup of the development environment and creation of train, validation and test splits is greatly simplified. Graphing, embedded within the application, illustrates the progress of model training, and SentencePiece is used for creating subword segmentation models. Hyperparameter customization is facilitated through an intuitive user interface, and a single-click model development approach has been implemented. Models developed by adaptNMT can be evaluated using a range of metrics, and deployed as a translation service within the application. To support eco-friendly research in the NLP space, a green report also flags the power consumption and kgCO$$_{2}$$ 2 emissions generated during model development. The application is freely available (http://github.com/adaptNMT).

show abstract

Section: Stochastic Nuancessupporting

confidence: 70%

adaptNMT: an open-source, language-agnostic development environment for neural machine translation

Lankford

Afli

Way

2023

Lang Resources & Evaluation

Self Cite

View full text Add to dashboard Cite

show abstract

“…Liang et al [118] tried to improve on the student model for the translation task by attempting to train a teacher using multiple sub-networks to produce various output variants. To improve upon NMT, Jooste et al [119] employed sequencelevel knowledge distillation on small-sized student models to distill knowledge from large teacher models. Quantization This is the method of compressing neural networks which are described as alleviating the precision of weights.…”

Section: Knowledge Distillationmentioning

confidence: 99%

Machine Translation Systems Based on Classical-Statistical-Deep-Learning Approaches

et al. 2023

View full text Add to dashboard Cite

Over recent years, machine translation has achieved astounding accomplishments. Machine translation has become more evident with the need to understand the information available on the internet in different languages and due to the up-scaled exchange in international trade. The enhanced computing speed due to advancements in the hardware components and easy accessibility of the monolingual and bilingual data are the significant factors that have added up to boost the success of machine translation. This paper investigates the machine translation models developed so far to the current state-of-the-art providing a solid understanding of different architectures with the comparative evaluation and future directions for the translation task. Because hybrid models, neural machine translation, and statistical machine translation are the types of machine translation that are utilized the most frequently, it is essential to have an understanding of how each one functions. A comprehensive comprehension of the several approaches to machine translation would be made possible as a result of this. In order to understand the advantages and disadvantages of the various approaches, it is necessary to conduct an in-depth comparison of several models on a variety of benchmark datasets. The accuracy of translations from multiple models is compared using metrics such as the BLEU score, TER score, and METEOR score.

show abstract

“…Due to the introduction of AI to NMT systems, the quality of MT outputs has significantly increased [10]. This has implications not only for professional translation but also in the classroom, as students start to shift their attention to MT results rather than to other translation strategies.…”

Section: Neural Machine Translationmentioning

confidence: 99%

The Challenges of Teaching and Assessing Technical Translation in an Era of Neural Machine Translation

et al. 2023

View full text Add to dashboard Cite

Teaching translation in higher education has undeniably been impacted by the innovations brought about by machine translation (MT), more particularly neural machine translation (NMT). This influence has become significantly more noticeable in recent years, as NMT technology progresses hand in hand with artificial intelligence. A case study supported by a questionnaire conducted among translation students (bachelor’s and master’s programmes at ISCAP) probed the degree of student satisfaction with CAT tools and revealed that they favour the use of MT in their translation practices, focusing their work on post-editing tasks rather than exploring other translation strategies and complementary resources. Although MT cannot be disregarded in translation programmes, as machine-generated translations make up an increasingly larger amount of a professional translator’s output, the widespread use of MT by students poses new challenges to translators’ training, since it becomes more difficult to assess students’ level of proficiency. Translation teachers must not only adapt their classroom strategies to accommodate these current translation strategies (NMT) but also, as intended by this study, find new, adequate methods of training and assessing students that go beyond regular translation assignments while still ensuring that students acquire the proper translation competence. Thus, as the use of NMT makes it considerably more challenging to assess a student’s level of translation competence, it is necessary to introduce other activities that not only allow students to acquire and develop their translation competence as defined in the EMT (European Masters in Translation) framework but also enable teachers to assess students more objectively. Hence, this article foregrounds a set of activities usually regarded as “indirect tasks” for technical translation courses that hopefully results in the development of student translation skills and competence, as well as provides more insights for teachers on how to more objectively assess students. It is possible, then, to conclude that these activities, such as different types of paraphrasing and error-detection tasks, may have the potential to encourage creative thinking and problem-solving strategies, giving teachers more resources to assess students’ level of translation competence.

show abstract

Knowledge Distillation: A Method for Making Neural Machine Translation More Efficient

Cited by 12 publications

References 17 publications

adaptNMT: an open-source, language-agnostic development environment for neural machine translation

adaptNMT: an open-source, language-agnostic development environment for neural machine translation

Machine Translation Systems Based on Classical-Statistical-Deep-Learning Approaches

The Challenges of Teaching and Assessing Technical Translation in an Era of Neural Machine Translation

Contact Info

Product

Resources

About