AuGPT: Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Models

Kulhánek, Jonáš; Hudeček, Vojtěch; Nekvinda, Tomáš; Dušek, Ondřej

doi:10.18653/v1/2021.nlp4convai-1.19

Cited by 27 publications

(26 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One idea that has gained particular attention is transfer learning -specifically, finding ways to leverage knowledge learned by pre-trained large language models (PLMs) for new tasks. PLMs have demonstrated impressive emerging conversational capabilities, enabling big performance improvements in various dialogue tasks (Brown et al, 2020;Shuster et al, 2022;Peng et al, 2022;Kulhánek et al, 2021). Particularly, PLMs have been prompted to augment existing conversational data (Chen et al, 2022;Mehri et al, 2022;Sahu et al, 2022).…”

Section: Triadic Conversations Dyadic Conversationsmentioning

confidence: 99%

“…However, prompt-based augmentation strategies are uncontrolled forms of generation, which may result in generation mistakes for labeled datasets (Sahu et al, 2022;Chen et al, 2022;Meng et al, 2022). In contrast, other recent studies have instead proposed language augmentation strategies that use complex, highly-controlled frameworks that often involve fine-tuning generators (Papangelis et al, 2021;Kulhánek et al, 2021;. Such complex augmentation frameworks require larger amounts of seed data to maintain a ground-truth language distribution (Rosenbaum et al, 2022b;Kim et al, 2021b), and are more costly than prompting PLMs (Chen et al, 2022).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

PLACES: Prompting Language Models for Social Conversation Synthesis

Chen¹,

Papangelis²,

Tao³

et al. 2023

Preprint

View full text Add to dashboard Cite

Collecting high quality conversational data can be very expensive for most applications and infeasible for others due to privacy, ethical, or similar concerns. A promising direction to tackle this problem is to generate synthetic dialogues by prompting large language models. In this work, we use a small set of expert-written conversations as incontext examples to synthesize a social conversation dataset using prompting 1 . We perform several thorough evaluations of our synthetic conversations compared to human-collected conversations. This includes various dimensions of conversation quality with human evaluation directly on the synthesized conversations, and interactive human evaluation of chatbots fine-tuned on the synthetically generated dataset. We additionally demonstrate that this prompting approach is generalizable to multiparty conversations, providing potential to create new synthetic data for multi-party tasks. Our synthetic multi-party conversations were rated more favorably across all measured dimensions compared to conversation excerpts sampled from a human-collected multi-party dataset.

show abstract

Section: Triadic Conversations Dyadic Conversationsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

PLACES: Prompting Language Models for Social Conversation Synthesis

Chen¹,

Papangelis²,

Tao³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…We denote several versions of the models provided in their paper as SeKnow-S2S(Single), SeKnow-S2S(Multiple), and SeKnow-GPT2. AuGPT (Kulhánek et al, 2021) uses modified training objectives and employs data augmentation to increase the diversity of generated utterances. HyKnow (Gao et al, 2021) first extends the dialogue state to query the database, and then uses all the information to generate a system response.…”

Section: End-to-end Modelmentioning

confidence: 99%

A Unified Dialogue Framework Based on Joint Generation of Dialogue Acts and Responses

Rong

et al. 2022

Preprint

View full text Add to dashboard Cite

Currently, building an end-to-end dialogue system for multi-domain task-oriented dialogue has some enormous challenges. Dialogue systems must obtain entire dialogue states from all relevant domains in order to respond correctly. However, multiple domains are involved in the actual dialogue process, which increases the difficulty of obtaining the dialogue state. In addition, dialogue systems process diverse information such as dialogue history, dialogue state, dialogue act, and database across domains, resulting in natural responses. These complex dialogue information brings greater difficulties to response generation. In this paper, we propose a novel unified dialogue framework for multi-domain dialogue in task-oriented dialog, including three modules: Encoder, Dialogue State Tracker and Multiple Decoders. First, encoder module encode all text input into continuous representations. Secondly, we train the dialogue state tracker module with a stacked-attention architecture.It learns information from slot-atten structure and domain-atten structure to track dialogue state. Then, multiple decoders module consists of act decoder structure and response decoder structure. It combines information from different textual inputs while modeling dialogue act. Finally, we jointly train all the above modules to generate system responses. We conducted extensive experiments on the dataset MultiWOZ. The experimental results show that our model achieves state-of-the-art results on evaluation metrics compared to models from previous work.

show abstract

“…Baselines Our framework is compared to other end-to-end unified TODS approaches that perform end-to-end TODS using a unified text-totext paradigm through a single generalized text (Su et al, 2022), Soloist (Peng et al, 2021), UBAR (Yang et al, 2021), AuGPT (Kulhánek et al, 2021), and Galaxy . In addition, as described in §3, we add the naive version of the LLM response generation approach which is fed by the full KB (RG naive ), as an additional baseline to better evaluate the effectiveness of our framework.…”

Section: Experiments Settingsmentioning

confidence: 99%

Incorporating Specific Knowledge into End-to-End Task-oriented Dialogue Systems

Wang

Cao

Jiang

et al. 2021

2021 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Large language models (LLMs) have been used for diverse tasks in natural language processing (NLP), yet remain under-explored for task-oriented dialogue systems (TODS), especially for end-to-end TODS. We present In-structTODS, a novel off-the-shelf framework for zero-shot end-to-end task-oriented dialogue systems that can adapt to diverse domains without fine-tuning. By leveraging LLMs, Instruct-TODS generates a proxy belief state that seamlessly translates user intentions into dynamic queries for efficient interaction with any KB. Our extensive experiments demonstrate that In-structTODS achieves comparable performance to fully fine-tuned TODS in guiding dialogues to successful completion without prior knowledge or task-specific data. Furthermore, a rigorous human evaluation of end-to-end TODS shows that InstructTODS produces dialogue responses that notably outperform both the gold responses and the state-of-the-art TODS in terms of helpfulness, informativeness, and humanness. Moreover, the effectiveness of LLMs in TODS is further supported by our comprehensive evaluations on TODS subtasks: dialogue state tracking, intent classification, and response generation. Code and implementations could be found here 1 . Anonymous. 2023. Nusawrites: Constructing highquality corpora for underrepresented and extremely low-resource languages. Anonymous preprint under review.

show abstract

AuGPT: Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Models

Cited by 27 publications

References 26 publications

PLACES: Prompting Language Models for Social Conversation Synthesis

PLACES: Prompting Language Models for Social Conversation Synthesis

A Unified Dialogue Framework Based on Joint Generation of Dialogue Acts and Responses

Incorporating Specific Knowledge into End-to-End Task-oriented Dialogue Systems

Contact Info

Product

Resources

About