Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters

Xu, Yan; Ishii, Etsuko; Liu, Zihan; Winata, Genta Indra; Su, Dan; Madotto, Andrea; Fung, Pascale

doi:10.18653/v1/2022.dialdoc-1.10

Cited by 18 publications

(17 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…S2S BERT replaces the Transformer Encoder with a pre-trained BERT (Devlin et al, 2019). KnowExpert (Xu et al, 2021) avoids the knowledge retrieval process and attempts to inject prior knowledge into the pre-trained language models for knowledge-grounded dialogue generation task. Essentially, KnowExpert stores knowledge in its parameters with lightweight adapters.…”

Section: A12 Our Supervised Methodsmentioning

confidence: 99%

Unsupervised Knowledge Selection for Dialogue Generation

Chen¹,

Chen²,

Meng³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

Knowledge selection is an important and challenging task which could provide the appropriate knowledge for informative dialogue generation. However, the needed gold knowledge label is difficult to collect in reality. In this paper, we study knowledge selection for dialogue generation in the unsupervised scenario and propose a novel Distilled Distant Supervision Loss (DDSL) to supervise knowledge selection when the gold knowledge label is unknown. Specifically, we first obtain an oracle knowledge label via distant supervision and then leverage knowledge distillation to alleviate the noisy labeling problem of distant supervision. Furthermore, we propose a pretraining-finetuning strategy to deal with the mismatch knowledge selection problem that models tend to select the mismatched knowledge for dialogue generation in the unsupervised setting and will cause the degeneration of knowledge-aware decoder. Experiments on two knowledge-grounded dialogue datasets show that our approach manages to select knowledge more accurately in the unsupervised setting and generates more informative responses, even outperforming many strong supervised baselines. 1

show abstract

Section: A12 Our Supervised Methodsmentioning

confidence: 99%

Unsupervised Knowledge Selection for Dialogue Generation

Chen¹,

Chen²,

Meng³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

show abstract

“…In order to perform the posterior sampling of knowledge selection during joint training, some works have proposed to separately train the posterior distribution model (Paranjape et al, 2022;Lian et al, 2019) or the posterior information prediction model (Chen et al, 2020). Very recently, SPI (Xu et al, 2023) has applied short-run MCMC (Erik et al, 2019) for posterior sampling on the collaborative latent spaces.…”

Section: Unsupervised Joint Trainingmentioning

confidence: 99%

“…In specific, they have treated the retrieved knowledge, document, or passage as an unobserved latent variable and adapt latent variable modeling based on approximated marginalization (e.g. top-k) Huang et al, 2021;Cai et al, 2023;Guu et al, 2020), reinforcement learning (Zhao et al, 2020;Zhang et al, 2022;Chen et al, 2022; or variational methods (Zhan et al, 2021;Paranjape et al, 2022;Lian et al, 2019;Kim et al, 2020;Chen et al, 2020;Xu et al, 2023). However, joint training of the retriever along with the generator under this latent variable modeling has some restrictions in utilizing the retriever.…”

Section: Introductionmentioning

confidence: 99%

Efficient Latent Variable Modeling for Knowledge-Grounded Dialogue Generation

Han,

Jo,

Nam

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Knowledge-grounded dialogue generation requires to first retrieve appropriate external knowledge based on a conversational context and then generate a response grounded on the retrieved knowledge. In general, these two sequential modules, a knowledge retriever and a response generator, have been separately trained by supervised data for each module. However, obtaining intermediate labels of the ground-truth knowledge is expensive and difficult especially in open-domain conversation. Latent variable modeling can circumvent it and enables a joint training without the knowledge supervision. In this paper, we propose an efficient algorithm for this latent variable modeling that is able to leverage a large amount of dialogue data. In specific, rather than directly training the complex retriever, we adapt a query generator with an off-the-shelf retriever, and the query generator and response generator are simultaneously trained over the latent variable of query. Moreover, we employ the evidence lower bound as a training objective and modify it to efficiently and robustly perform the joint training. Experimental results on diverse knowledge-grounded dialogue datasets show that the proposed algorithm achieves state-ofthe-art performances even without the use of the annotated knowledge while maintaining the efficiency and scalability.

show abstract

“…Furthermore, it also has been pointed out that using a knowledge base could reduce the problem of hallucinations (Dziri et al, 2021). Another research line tends to compress knowledge into model parameters, either by training set augmentation with template-based method (Madotto et al, 2020) or using neural architectures as domain-specific adapters (Xu et al, 2021).…”

Section: Grounded Dialogue Generationmentioning

confidence: 99%

On Controlling Fallback Responses for Grounded Dialogue Generation

Lu¹,

Lam²,

Chen³

et al. 2022

Findings of the Association for Computational Linguistics: ACL 2022

View full text Add to dashboard Cite

Dialogue agents can leverage external textual knowledge to generate responses of a higher quality. To our best knowledge, most existing works on knowledge grounded dialogue settings assume that the user intention is always answerable. Unfortunately, this is impractical as there is no guarantee that the knowledge retrievers could always retrieve the desired knowledge. Therefore, this is crucial to incorporate fallback responses to respond to unanswerable contexts appropriately while responding to the answerable contexts in an informative manner. We propose a novel framework that automatically generates a control token with the generator to bias the succeeding response towards informativeness for answerable contexts and fallback for unanswerable contexts in an endto-end manner. Since no existing knowledge grounded dialogue dataset considers this aim, we augment the existing dataset with unanswerable contexts to conduct our experiments. Automatic and human evaluation results indicate that naively incorporating fallback responses with controlled text generation still hurts informativeness for answerable context. In contrast, our proposed framework effectively mitigates this problem while still appropriately presenting fallback responses to unanswerable contexts. Such a framework also reduces the extra burden of the additional classifier and the overheads introduced in the previous works, which operates in a pipeline manner. 1

show abstract

Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters

Cited by 18 publications

References 39 publications

Unsupervised Knowledge Selection for Dialogue Generation

Unsupervised Knowledge Selection for Dialogue Generation

Efficient Latent Variable Modeling for Knowledge-Grounded Dialogue Generation

On Controlling Fallback Responses for Grounded Dialogue Generation

Contact Info

Product

Resources

About