Robust Belief State Space Representation for Statistical Dialogue Managers Using Deep Autoencoders

Abstract

Statistical Dialogue Systems (SDS) have proved their humongous potential over the past few years. However, the lack of efficient and robust representations of the belief state (BS) space refrains them from revealing their full potential. There is a great need for automatic BS representations, which will replace the old hand-crafted, variable-length ones. To tackle those problems, we introduce a novel use of Autoencoders (AEs). Our goal is to obtain a low-dimensional, fixed-length, and compact, yet robust representation of the BS space. We investigate the use of dense AE, Denoising AE (DAE) and Variational Denoising AE (VDAE), which we combine with GP-SARSA to learn dialogue policies in the PyDial toolkit. In this framework, the BS is normally represented in a relatively compact, but still redundant summary space which is obtained through a heuristic mapping of the original master space. We show that all the proposed AE-based representations consistently outperform the summary BS representation. Especially, as the Semantic Error Rate (SER) increases, the DAE/VDAE-based representations obtain state-of-the-art and sample efficient performance.

Variational Denoising Autoencoders and Least-Squares Policy Iteration for Statistical Dialogue Managers

“…Similar are the findings when comparing with [8], where the sumBS was used with different RL algorithms on the same domains and environments. A comparison is also possible with [14], where the max accuracy for 45% SER on the CR domain is 52.9% compared to 93.6% in this work, and with [19], which shows that VDAE-LSPI is on average more than 5% better compared to GP-SARSA based approaches.…”

“…A second contribution is that our approach capitilizes on a new state-space representation based on noise-robust and compact encodings obtained through unsupervised training of deep Variational Denoising Auto-Encoders (VDAE). In contrast to our recent work in [19], here we combine these encodings with LSPI instead of GP-SARSA. Our scheme proves its efficiency under a variety of noisy conditions, both matched and mismatched, outperforming the current state-ofthe-art.…”

“…The weights are initially approximated with the Temporal Difference (TD) algorithm and the selected features are then used in the framework of the sample-efficient Least-Squares Policy Iteration (LSPI) [18]. Recently, in [19], state-of-the-art performance was obtained using deep denoising autoencoders to produce robust BS encodings for the GP-SARSA learning algorithm.…”