Evolving transformer architecture for neural machine translation

Feng, Mingbin; Liu, Dayiheng; Sun, Yanan

doi:10.1145/3449726.3459441

Cited by 6 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…generation has wide range of applications in deep learning, which can be categorized into 18 groups as shown in Figure 9. We have found 10 out of 90 papers have main purpose to balance the dataset [6], [51], [69], [85], [86], [118], [121], [132], out 90 papers have worked on data to text [48], [56], [65], [83], [103], [105], [111], [129], and speech to text [68], [84], [101], [106]- [108], [113], [116], respectively.7 papers have worked on script writing [3], [17], [58], [61], [86], 5 papers have worked on machine translation [10], [11], [57], [88], [104], [123]. Apart from these, 4 papers have worked on text summarization [50], [88], [100], [130] and 2 papers [1], [91] have worked on abstract meaning representation (AMR)-AMR to text goal is to generate sentences from abstract meaning representation graphs and its seq2seq or graph2seq problem.…”

Section: Metrics Groupmentioning

confidence: 99%

“…It is the most widely used variant of text generation models. Language translation belong to this type of text generation model [5] Above all, deep learning has contributed immensely to different aspects of natural language generation for various tasks including, dataset balancing [6], [7], next word prediction & text suggestion in chatting, generation of answers to questions in question answering system, in chatbots [8], [9], machine learning translation [10], [11], text summarization [12]- [14], text classification [15], [16], text generation for topic modeling [17], dialogue generation [18], sentiment analysis [19], [20], poetry writing [21], script writing for movies [1], [22], and others.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Systematic Literature Review on Text Generation Using Deep Neural Network Models

et al. 2022

View full text Add to dashboard Cite

In recent years, significant progress has been made in text generation. The latest text generation models are revolutionizing the domain by generating human-like text. It has gained wide popularity recently in many domains like news, social networks, movie script writing, poetry composition, to name a few. The application of text generation in various fields has resulted in a lot of interest from the scientific community in this area. To the best of our knowledge, there is a lack of extensive review and up-to-date body of knowledge of text generation deep learning models. Therefore, this paper aims to bring together all the relevant work in a systematic mapping study highlighting key contributions from various researchers over the years, focusing on the past, present, and future trends. In this work, we have identified 90 primary studies from 2015 to 2021 employing the PRISMA framework. We also identified research gaps that are further needed to be explored by the research community. In the end, we provide some future directions for researchers and guidelines for practitioners based on the findings of this review.

show abstract

Section: Metrics Groupmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Systematic Literature Review on Text Generation Using Deep Neural Network Models

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Most NAS research efforts have centered around the computer vision task of image classification and only recently have other modalities, such as the rapidly growing field of language modeling or language translation, been investigated in detail [10,11]. Moreover, understanding how NAS approaches generalize and perform across modalities and tasks has not been studied in depth.…”

Section: Introductionmentioning

confidence: 99%

Accelerating Neural Architecture Exploration Across Modalities Using Genetic Algorithms

Cummings¹,

Sridhar²,

Anthony³

et al. 2022

Preprint

View full text Add to dashboard Cite

Neural architecture search (NAS), the study of automating the discovery of optimal deep neural network architectures for tasks in domains such as computer vision and natural language processing, has seen rapid growth in the machine learning research community. While there have been many recent advancements in NAS, there is still a significant focus on reducing the computational cost incurred when validating discovered architectures by making search more efficient. Evolutionary algorithms, specifically genetic algorithms, have a history of usage in NAS and continue to gain popularity versus other optimization approaches as a highly efficient way to explore the architecture objective space. Most NAS research efforts have centered around computer vision tasks and only recently have other modalities, such as the rapidly growing field of natural language processing, been investigated in depth. In this work, we show how genetic algorithms can be paired with lightly trained objective predictors in an iterative cycle to accelerate multi-objective architectural exploration in a way that works in the modalities of both machine translation and image classification.

show abstract

A Systematic survey on automated text generation tools and techniques: application, evaluation, and challenges

Goyal¹,

Kumar²,

Singh³

2023

Multimed Tools Appl

View full text Add to dashboard Cite

Evolving transformer architecture for neural machine translation

Cited by 6 publications

References 7 publications

A Systematic Literature Review on Text Generation Using Deep Neural Network Models

A Systematic Literature Review on Text Generation Using Deep Neural Network Models

Accelerating Neural Architecture Exploration Across Modalities Using Genetic Algorithms

A Systematic survey on automated text generation tools and techniques: application, evaluation, and challenges

Contact Info

Product

Resources

About