Co-adaptation in Spoken Dialogue Systems

Chandramohan, Senthilkumar; Geist, Matthieu; Lefèvre, Fabrice; Pietquin, Olivier

doi:10.1007/978-1-4614-8280-2_31

Cited by 11 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To address the time-consuming development for simulator policy, recent studies [10,26,27] proposed a one-to-one dialogue model where a dialogue manager and a user simulator were optimized concurrently. Different from the above studies, our proposed MADM applies the reward shaping technique [11] based on the adjacency pairs in conversational analysis [12], which can help the cooperative policies learn from scratch quickly.…”

Section: Related Workmentioning

confidence: 99%

“…Different from the above studies, our proposed MADM applies the reward shaping technique [11] based on the adjacency pairs in conversational analysis [12], which can help the cooperative policies learn from scratch quickly. By the method of reward shaping, our proposed MADM avoids running a learning algorithm multiple times in a study [26] and collects the corpora in studies [10,27].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Cooperative Multi-Agent Reinforcement Learning with Conversation Knowledge for Dialogue Management

2020

View full text Add to dashboard Cite

Dialogue management plays a vital role in task-oriented dialogue systems, which has become an active area of research in recent years. Despite the promising results brought from deep reinforcement learning, most of the studies need to develop a manual user simulator additionally. To address the time-consuming development of simulator policy, we propose a multi-agent dialogue model where an end-to-end dialogue manager and a user simulator are optimized simultaneously. Different from prior work, we optimize the two-agents from scratch and apply the reward shaping technology based on adjacency pairs constraints in conversational analysis to speed up learning and to avoid the derivation from normal human-human conversation. In addition, we generalize the one-to-one learning strategy to one-to-many learning strategy, where a dialogue manager can be concurrently optimized with various user simulators, to improve the performance of trained dialogue manager. The experimental results show that one-to-one agents trained with adjacency pairs constraints can converge faster and avoid derivation. In cross-model evaluation with human users involved, the dialogue manager trained in one-to-many strategy achieves the best performance. from scratch without supervised initializing process. For user simulator reward function, we use the reward shaping technique [11] based on the adjacency pairs in conversational analysis [12] to make the simulator learn real user behaviors quickly. In addition, we generalize the one-to-one learning strategy to one-to-many learning strategy where a dialogue manager cooperates with various user simulators to improve the performance of trained dialogue manager. We obtain these various user simulators through changing the adjacency pairs settings, and then we mixture them with a dialogue manager to optimize the cooperative policies via multi-agent reinforcement learning.Compared with MADM without the constraints, MADM trained with adjacency pairs constraints can converge faster and avoid derivation from normal human-human conversation. The experimental results also show that the dialogue manager trained with one-to-many strategy achieves the best performance in cross-model evaluation with human users involved. To summary, our main contributions in this work are three-fold:

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Cooperative Multi-Agent Reinforcement Learning with Conversation Knowledge for Dialogue Management

2020

View full text Add to dashboard Cite

show abstract

“…Another bottleneck in domain adaptation that has not yet been resolved is the need to predefine all actions and slots prior to learning. It is desirable for a policy to be able to dynamically adapt to new concepts discovered throughout an interaction, similar to how humans continuously evolve and learn through communication (Chandramohan et al 2014), moving further beyond multi‐domain dialogue systems to open‐domain ones.…”

Section: Adaptationmentioning

confidence: 99%

Adaptable Conversational Machines

Lubis¹,

Heck²,

Niekerk³

et al. 2020

AI Magazine

View full text Add to dashboard Cite

In recent years we have witnessed a surge in machine learning methods that provide machines with conversational abilities. Most notably, neural-network–based systems have set the state of the art for difficult tasks such as speech recognition, semantic understanding, dialogue management, language generation, and speech synthesis. Still, unlike for the ancient game of Go for instance, we are far from achieving human-level performance in dialogue. The reasons for this are numerous. One property of human–human dialogue that stands out is the infinite number of possibilities of expressing oneself during the conversation, even when the topic of the conversation is restricted. A typical solution to this problem was scaling-up the data. The most prominent mantra in speech and language technology has been “There is no data like more data.” However, the researchers now are focused on building smarter algorithms — algorithms that can learn efficiently from just a few examples. This is an intrinsic property of human behavior: an average human sees during their lifetime a fraction of data that we nowadays present to machines. A human can even have an intuition about a solution before ever experiencing an example solution. The human-inspired ability to adapt may just be one of the keys in pushing dialogue systems toward human performance. This article reviews advancements in dialogue systems research with a focus on the adaptation methods for dialogue modeling, and ventures to have a glance at the future of research on adaptable conversational machines.

show abstract

“…We argue that if the human operator may not be optimally acting to maximize the users’ satisfaction, the users are unconsciously trying to optimize their satisfaction when interacting with a machine. IRL could, therefore, be used to learn the internal (non-observable) reward function that users naturally try to maximize (Chandramohan et al ., 2011).…”

Section: Future Directionsmentioning

confidence: 99%

A survey on metrics for the evaluation of user simulations

Pietquin

Hastie

2012

The Knowledge Engineering Review

Self Cite

View full text Add to dashboard Cite

International audienceUser simulation is an important research area in the field of spoken dialogue systems (SDSs) because collecting and annotating real human-machine interactions is often expensive and time-consuming. However, such data are generally required for designing, training and assessing dialogue systems. User simulations are especially needed when using machine learning methods for optimizing dialogue management strategies such as Reinforcement Learning, where the amount of data necessary for training is larger than existing corpora. The quality of the user simulation is therefore of crucial importance because it dramatically influences the results in terms of SDS performance analysis and the learnt strategy. Assessment of the quality of simulated dialogues and user simulation methods is an open issue and, although assessment metrics are required, there is no commonly adopted metric. In this paper, we give a survey of User Simulations Metrics in the literature, propose some extensions and discuss these metrics in terms of a list of desired features

show abstract

Co-adaptation in Spoken Dialogue Systems

Cited by 11 publications

References 11 publications

Cooperative Multi-Agent Reinforcement Learning with Conversation Knowledge for Dialogue Management

Cooperative Multi-Agent Reinforcement Learning with Conversation Knowledge for Dialogue Management

Adaptable Conversational Machines

A survey on metrics for the evaluation of user simulations

Contact Info

Product

Resources

About