Neural Machine Translation with Key-Value Memory-Augmented Attention

Meng, Fandong; Tu, Zhaopeng; Cheng, Yong; Wu, Haiyang; Zhai, Junjie; Yang, Yuekui; Wang, Di

doi:10.24963/ijcai.2018/357

Cited by 13 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dialog History Memory Update: Dialog history memory update is inspired by Fandong Meng et al [19][20] , where we use both adding and forgetting to update the dialog history memory. The forget operation determines how much of the current message should be removed from the dialog history memory, and similarly, the add operation determines how much of the current message should be written to the dialog history memory as an addition.…”

Section: Dialog History Manager Modulementioning

confidence: 99%

Dialog history management for end-to-end task-oriented dialog systems

Nie

Zhang

et al. 2023

Fifth International Conference on Computer Information Science and Artificial Intelligence (CISAI 2022)

View full text Add to dashboard Cite

End-to-end task-oriented dialogue systems rely heavily on an understanding of the dialog history. This often faces the challenge of inferring which dialog history information is critical to generating responses. In this paper, we address this challenge by leveraging a dialog history manager component that dynamically focuses on dialog history memory. It performs multiple add and forget operations by fusing an enhanced entity representation of dialog history and knowledge base (KB) information as queries, remembering entities relevant to responses and filtering out unimportant information. Experimental results on an open task-oriented dialogue dataset show that our model outperforms the baseline system in terms of effectiveness and produces contextually consistent responses.

show abstract

Section: Dialog History Manager Modulementioning

confidence: 99%

Dialog history management for end-to-end task-oriented dialog systems

Nie

Zhang

et al. 2023

Fifth International Conference on Computer Information Science and Artificial Intelligence (CISAI 2022)

View full text Add to dashboard Cite

show abstract

“…t,j v j , where v j is the j-th memory slot in V. Dialog State Memory Updating Inspired by the read-write operations (Meng et al, 2016;Meng et al, 2018), we define two types of operations for updating the dialog state memory: FORGET and ADD. FORGET is analogous to the forget gate in GRU, which determines the information to be removed from memory slots.…”

Section: Dialog History Memory Readingmentioning

confidence: 99%

Dual Dynamic Memory Network for End-to-End Multi-turn Task-oriented Dialog Systems

Wang

Liu

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

Existing end-to-end task-oriented dialog systems struggle to dynamically model long dialog context for interactions and effectively incorporate knowledge base (KB) information into dialog generation. To conquer these limitations, we propose a Dual Dynamic Memory Network (DDMN) for multi-turn dialog generation, which maintains two core components: dialog memory manager and KB memory manager. The dialog memory manager dynamically expands the dialog memory turn by turn and keeps track of dialog history with an updating mechanism, which encourages the model to filter irrelevant dialog history and memorize important newly coming information. The KB memory manager shares the structural KB triples throughout the whole conversation, and dynamically extracts KB information with a memory pointer at each turn. Experimental results on three benchmark datasets demonstrate that DDMN significantly outperforms the strong baselines in terms of both automatic evaluation and human evaluation. Our code is available at https://github.com/siat-nlp/DDMN.

show abstract

“…Tu et al (2016), Mi et al (2016) and Li et al (2018b) employ coverage vector or coverage ratio to indicate the lexical-level coverage of source words. Meng et al (2018) influence the attentive vectors by translated/untranslated information. Our work mainly follows the path of Zheng et al (2018), which introduce two extra recurrent layers in the decoder to maintain the representations of the past and future translation contents.…”

Section: Related Workmentioning

confidence: 99%

Dynamic Past and Future for Neural Machine Translation

Zheng¹,

Huang²,

Tu³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

Self Cite

View full text Add to dashboard Cite

Previous studies have shown that neural machine translation (NMT) models can benefit from explicitly modeling translated (PAST) and untranslated (FUTURE) source contents as recurrent states (Zheng et al., 2018). However, this less interpretable recurrent process hinders its power to model the dynamic updating of PAST and FUTURE contents during decoding. In this paper, we propose to model the dynamic principles by explicitly separating source words into groups of translated and untranslated contents through parts-to-wholes assignment. The assignment is learned through a novel variant of routing-by-agreement mechanism (Sabour et al., 2017), namely Guided Dynamic Routing, where the translating status at each decoding step guides the routing process to assign each source word to its associated group (i.e., translated or untranslated content) represented by a capsule, enabling translation to be made from holistic context. Experiments show that our approach achieves substantial improvements over both RNMT and Transformer by producing more adequate translations. Extensive analysis demonstrates that our method is highly interpretable, which is able to recognize the translated and untranslated contents as expected. 1

show abstract

Neural Machine Translation with Key-Value Memory-Augmented Attention

Cited by 13 publications

References 0 publications

Dialog history management for end-to-end task-oriented dialog systems

Dialog history management for end-to-end task-oriented dialog systems

Dual Dynamic Memory Network for End-to-End Multi-turn Task-oriented Dialog Systems

Dynamic Past and Future for Neural Machine Translation

Contact Info

Product

Resources

About