A Versatile Adaptive Curriculum Learning Framework for Task-oriented Dialogue Policy Learning

Abstract

Training a deep reinforcement learning-based dialogue policy with brute-force random sampling is costly. A new training paradigm was proposed to improve learning performance and efficiency by combining curriculum learning. However, attempts in the field of dialogue policy are very limited due to the lack of reliable evaluation of difficulty scores of dialogue tasks and the high sensitivity to the mode of progression through dialogue tasks. In this paper, we present a novel versatile adaptive curriculum learning (VACL) framework, which presents a substantial step toward applying automatic curriculum learning on dialogue policy tasks. It supports evaluating the difficulty of dialogue tasks only using the learning experiences of dialogue policy and skip-level selection according to their learning needs to maximize the learning efficiency. Moreover, an attractive feature of VACL is the construction of a generic, elastic global curriculum while training a good dialogue policy that could guide different dialogue policy learning without extra effort on re-training. The superiority and versatility of VACL are validated on three public dialogue datasets.

Cooper: Coordinating Specialized Agents towards a Complex Dialogue Goal

“…In the past few years, there has been growing interest in dialogue generation tasks with complex objectives, such as negotiation (Lewis et al 2017;He et al 2018;Zhou et al 2019b), persuasion (Wang et al 2019;Li et al 2020;Samad et al 2022), and emotional support (Liu et al 2021a;Peng et al 2022;Xu, Meng, and Wang 2022;Zhao et al 2023b).…”

“…In real scenarios, the guidelines for these challenging dialogue tasks usually recommend breaking down the complex goals into multiple aspects and jointly promoting them to work towards the broad objective (Petty et al 1986;Fershtman 1990;Hill 2009). More recently, several works have applied LLMs to complex goal-oriented dialogues, by directly prompting the LLM to generate utterances (Zhao et al 2023a;Deng et al 2023a) or further improving the performance via iterative revision (Fu et al 2023). Current LLMs exhibit remarkable improvement compared to the previous methods on these tasks, but it is also found that they tend to lack a larger picture of the overall dialogue progression and fail to achieve the dialogue objective strategically through multi-turn interactions (Deng et al 2023a).…”

“…Current LLMs exhibit remarkable improvement compared to the previous methods on these tasks, but it is also found that they tend to lack a larger picture of the overall dialogue progression and fail to achieve the dialogue objective strategically through multi-turn interactions (Deng et al 2023a). For example, on the task of ESC, they often continuously offer coping suggestions and overlook the critical process of exploring the user's situation and expressing empathy (Zhao et al 2023a).…”

“…Compared to traditional service-focused goal-oriented dialogue systems (Rieser and Moore 2005;Boyer et al 2011;Wen et al 2016;Liu et al 2022), these tasks require much more sophisticated strategic reasoning and communication skills. Recent studies show that even state-of-the-art Large Language Models (LLMs) struggle with these tasks, where they exhibit weak awareness of the overall dialogue progression and fail to accomplish a complex dialogue goal through multiturn interactions strategically (Zhao et al 2023a). Moreover, another major challenge lies in the difficulty of objectively measuring the achievement of such complex dialogue goals in a quantifiable and reliable way.…”

Continuous-Speech Spoken Dialogue System Based on Joint Decision-Making of Audio and Semantics

“…If "it's burning hot, turn on the air conditioner for me" is played on TV, the voice may be parsed into the intention of controlling the air conditioner. However, it's not spoken by a real human, and thus shall be rejected [7]. Therefore, it is rather difficult for non-real-human sounds to be recognized at the level of text semantics.…”