When to Pre-Train Graph Neural Networks? From Data Generation Perspective!

Cao, Yuxuan; Xu, Jiarong; Yang, Carl; Wang, Jiaan; Zhang, Yunchao; Wang, Chunping; Chen, L; Yang, Yang

doi:10.1145/3580305.3599548

Cited by 4 publications

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Knowledge-Grounded Dialogue (KGD) aims to generate an informative response based on a given dialogue context and external knowledge to improve the usefulness and meaningfulness of the generated responses (Ghazvininejad et al 2017;Zhou et al 2018;Liu et al 2019;Kim, Ahn, and Kim 2020;Li et al 2021;Rashkin et al 2021;Wu et al 2022;Sun et al 2023). As for the choice of knowledge source, structural knowledge graphs (KGs) are proven options (Moon et al 2019;Jung, Son, and Lyu 2020;Wu et al 2022), which consist of a lot of knowledge facts that are frequently used in daily life (Zheng et al 2021;Zhang et al 2022;Cao et al Have you ever heard about Leonardo? Oh, absolutely!…”

Section: Introductionmentioning

confidence: 99%

Improving the Robustness of Knowledge-Grounded Dialogue via Contrastive Learning

Wang,

Qu,

Wang

et al. 2024

AAAI

View full text Add to dashboard Cite

Knowledge-grounded dialogue (KGD) learns to generate an informative response based on a given dialogue context and external knowledge (e.g., knowledge graphs; KGs). Recently, the emergence of large language models (LLMs) and pre-training techniques has brought great success to knowledge-grounded dialogue. However, when building KGD systems in real applications, there are various real-world noises that are inevitable to face. For example, the dialogue context might involve perturbations such as misspellings and abbreviations. In addition, KGs typically suffer from incompletion and also might contain erroneous and outdated facts. Such real-world noises pose a challenge to the robustness of KGD systems and hinder their applications in the real world. In this paper, we propose an entity-based contrastive learning framework for improving the robustness of KGD. Specifically, we make use of the entity information in a KGD sample to create both its positive and negative samples which involve semantic-irrelevant and semantic-relevant perturbations, respectively. The contrastive learning framework ensures the KGD model is aware of these two types of perturbations, thus could generate informative responses with the potentially noisy inputs in real applications. Experimental results on three widely-used benchmark datasets show that our method achieves new state-of-the-art performance in terms of automatic evaluation scores, verifying its effectiveness and potentiality. Furthermore, we show that our method is able to generate better responses than comparison models in both the noisy and the few-shot settings.

show abstract