Adversarial Bandit for online interactive active learning of zero-shot spoken language understanding

Abstract

Many state-of-the-art solutions for the understanding of speech data have in common to be probabilistic and to rely on machine learning algorithms to train their models from large amount of data. The difficulty remains in the cost of collecting and annotating such data. Another point is the time for updating an existing model to a new domain. Recent works showed that a zero-shot learning method allows to bootstrap a model with good initial performance. To do so, this method relies on exploiting both a small-sized ontological description of the target domain and a generic word-embedding semantic space for generalization. Then, this framework has been extended to exploit user feedbacks to refine the zero-shot semantic parser parameters and increase its performance online. In this paper, we propose to drive this online adaptive process with a policy learnt using the Adversarial Bandit algorithm Exp3. We show, on the second Dialog State Tracking Challenge (DSTC2) datasets, that this proposition can optimally balance the cost of gathering valuable user feedbacks and the overall performance of the spoken language understanding module.Index Terms-Spoken language understanding, zero-shot learning, bandit problem, out-of-domain training data, online adaptation.

Joint On-line Learning of a Zero-shot Spoken Semantic Parser and a Reinforcement Learning Dialogue Manager

“…An on-line adaptation strategy is adopted, as presented in [11] and briefly recalled here. In this approach, at each dialogue iteration, the system chooses an adaptation action i t ∈ I and uses the user feedback to update K.…”

“…In this system could have been implemented the most sample-efficient learning algorithms [8], from which on-line learning with direct interactions with the user could have been proposed [9]. More recently on-line learning has been generalized to the input/output modules, SP and NLG, with protocols to control the cost of such operations during the system development (as in [10,11,12]). In this work it is our first attempt to combine the on-line learning of SP and DM in a single phase of development.…”

“…Indeed, only the ontological description of the target domain and generic word embedding features (learned from freely available and general purpose data) are required to initiate the model. On top of that an active learning strategy based on an adversarial bandit has been proposed [11] in order to train ZSSP with a light and controlled supervision from the users.…”

Online Adaptation of an Attention-Based Neural Network for Natural Language Generation

“…In this respect, a reinforcement learning approach based on an adversarial bandit scheme is applied [14]. If this approach has previously been used in dialogue systems for language understanding [15,16], we propose a protocol to adapt the RNN-based model on new utterances that vary from the training dataset, taking into account the cost it implies for the user to give these examples.…”

Integration and evaluation of social competences such as humor in an artificial interactive agent

“…Lately, stochastic approaches made a breakthrough in natural language processing and are now used by state-of-the-art systems for the SLU (e.g. [3,6]) and the dialog manager (e.g. [4,5,7]).…”