Adversarial training and decoding strategies for end-to-end neural conversation models

Hori, Takaaki; Wang, Wen; Koji, Yusuke; Hori, Chiori; Harsham, Bret; Hershey, John R.

doi:10.1016/j.csl.2018.08.006

Cited by 12 publications

(7 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Automat ic based Metric F1-Score [89], [114], [106], [80], [62], [100] Precision [105], [106], [80], [62] Recall [105], [106], [80], [62] Accuracy [92], [114], [126], [105], [124], [80], [ 93], [62], [100], [76], [56], [74] PPL [89], [102], [94], [92], [98], [124], [12 6], [39], [101], [85] BLEU [79], [83], [94], [81], [92], [95], [84], [ 126], [127], [86], [101], [49], [40],[4 0],…”

Section: Categori Zation Metrics Articlesmentioning

confidence: 99%

English and Arabic Chatbots: A Systematic Literature Review

Alsheddi¹,

Alhenaki²

2022

IJACSA

View full text Add to dashboard Cite

In recent years, the availability of chatbot applications has increased substantially with the advancement of artificial intelligence techniques, and research efforts have been active in the English language, which presents state-of-the-art solutions. However, despite the popularity of the Arabic language, its research community is still in an immature stage. Therefore, the main objective of this systematic literature review is studying state-of-the-art researchfor both the English and Arabic languagesto answer the proposed research questions regarding the development approaches, application domains, evaluation metrics, and development challenges of chatbot applications. The findings show that researchers have devoted more attention to the education domain using retrieval-based approaches while the generation-based approach has grown in popularity recently for providing new responses tasks. Whereas the hybrid approach for ranking multi-possible responses of combining both previous approaches shows a performance improvement. Besides, most metrics used to evaluate chatbot performance are human-based, followed by bilingual evaluation understudy and accuracy metrics. However, defining a common framework for evaluating chatbots remains a challenge. Finally, the open problems and future directions are highlighted to help in developing chatbots with minimal human interference to simulate natural conversations.

show abstract

Section: Categori Zation Metrics Articlesmentioning

confidence: 99%

English and Arabic Chatbots: A Systematic Literature Review

Alsheddi¹,

Alhenaki²

2022

IJACSA

View full text Add to dashboard Cite

show abstract

“…Prior to the training of the dialogue system, we pretrain the word vector matrix in the same corpus that will be used later. Following the work of [14] and [9], we also pretrain the generator using the MLE criteria, and the discriminator with the responses generated by the pretrained generator and with responses from the corpus. In order to stabilize the rest of the training process and to avoid the catastrophic forgetting phenomenon of the discriminator, each time we sample a response of the generator to a given input, we add it to a corpus of generator's turns C D .…”

Section: Training Proceduresmentioning

confidence: 99%

“…We finally repeat this process of training the generator, adding samples to C D and training the discriminator, but this time training the corpus with the MLE criteria. This approach is also taken in [14] and [9], and it aims at stabilizing the training process. In order to further stabilize it, we reduce the learning rate of the training optimizer throughout the global iterations.…”

Section: Training Proceduresmentioning

confidence: 99%

“…[5] and [11] experiment with training discriminators that could measure the quality of the utterances generated by chatbots. On the other hand [14] and [9] go a step further and train neural dialogue systems via adversarial learning, but with the drawback that they make use of reinforcement learning instead of gradient-based optimization methods. This is due to text being represented as a sequence of discrete tokens, which breaks the differentiability of the discriminator's output with respect to the generator's parameters, as explained in Section 3.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Differentiable Generative Adversarial Network for Open Domain Dialogue

Zorrilla

Velasco

Torres

2021

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

This work presents a novel methodology to train open domain neural dialogue systems within the framework of Generative Adversarial Networks with gradient based optimization methods. We avoid the non-differentiability related to textgenerating networks approximating the word vector corresponding to each generated token via a top-k softmax. We show that a weighted average of the word vectors of the most probable tokens computed from the probabilities resulting of the top-k softmax leads to a good approximation of the word vector of the generated token. Finally we demonstrate through a human evaluation process that training a neural dialogue system via adversarial learning with this method successfully discourages it from producing generic responses. Instead it tends to produce more informative and variate ones.

show abstract

“…Xehetasunetan sartu gabe, sare diskriminatzailearen irteera ez da diferentziagarria sare sortzailearen parametroekiko, sortzaileak sortutako hitzak diskretuak dira eta [11]. Errefortzu bidezko ikasketa erabili daiteke gradienteetan oinarritutako metodoen ordez [12,13], baina horrek entrenamenduaren konbergentzia zaildu dezake [14]. Beste aukera bat straight-through Gumbel-softmax [15,16] zenbateslearen bidez gradientearen hurbilketa bat egitea da, [17] eta [18] laneetan erakusten duten moduan.…”

Section: Arloko Egoera Eta Ikerketaren Helburuakunclassified

Euskarazko elkarrizketa sistema automatikoa sare neuronalen bidez

López-Zorrilla

Vázquez

Justo

2020

EKAIA

View full text Add to dashboard Cite

Lan honetan sare neuronalen bidez euskaraz hitz egiten ikasten duen elkarrizketa sistema automatiko bat aurkezten dugu. Horretarako, turingen testaren ideia era konputazionalean inplementatzen duten sare neuronal sortzaile aurkariak erabili ditugu. Normalean erabiltzen diren ingelesezko corpusak baino bi magnitude ordena txikiagoa den euskarazko corpus batekin halako sareak doitzea badagoela frogatzen dugu. Amaitzeko, euskararen morfologia kontuan hartzen duen aurreprozesamendua erabiltzea komenigarria dela erakusten dugu. Sare neuronaletan oinarrituta dagoen euskarazko lehen elkarrizketa sistema aurkezten dugu.

show abstract

Adversarial training and decoding strategies for end-to-end neural conversation models

Cited by 12 publications

References 10 publications

English and Arabic Chatbots: A Systematic Literature Review

English and Arabic Chatbots: A Systematic Literature Review

A Differentiable Generative Adversarial Network for Open Domain Dialogue

Euskarazko elkarrizketa sistema automatikoa sare neuronalen bidez

Contact Info

Product

Resources

About