Curriculum-Based Transfer Learning for an Effective End-to-End Spoken Language Understanding and Domain Portability

Abstract

We present an end-to-end approach to extract semantic concepts directly from the speech audio signal. To overcome the lack of data available for this spoken language understanding approach, we investigate the use of a transfer learning strategy based on the principles of curriculum learning. This approach allows us to exploit out-of-domain data that can help to prepare a fully neural architecture. Experiments are carried out on the French MEDIA and PORTMEDIA corpora and show that this end-toend SLU approach reaches the best results ever published on this task. We compare our approach to a classical pipeline approach that uses ASR, POS tagging, lemmatizer, chunker... and other NLP tools that aim to enrich ASR outputs that feed an SLU text to concepts system. Last, we explore the promising capacity of our end-to-end SLU approach to address the problem of domain portability.

A Data Efficient End-to-End Spoken Language Understanding Architecture

“…4 SLU performances are given in Table 3. Our results can be compared with some previous works [4,3]. We note however that results reported in [4,3] are obtained with models trained with much more data exploiting NER tasks with transfer learning.…”

“…We evaluate both ASR (Word Error Rate) and SLU (Concept Error Rate) results on MEDIA corpus (Dev and Test). State-of-the-art Models E2E SLU [4] 300h 30.1 27.0 E2E Baseline [3] 41.5h -39.8 E2E SLU [3] 500h -23.7 E2E SLU + curr. [3] 500h -16.4 ASR results are presented in Table 2.…”

“…State-of-the-art Models E2E SLU [4] 300h 30.1 27.0 E2E Baseline [3] 41.5h -39.8 E2E SLU [3] 500h -23.7 E2E SLU + curr. [3] 500h -16.4 ASR results are presented in Table 2. Together with the character and the 2-stage models, we show performance from a model decoding directly tokens.…”

“…First SLU systems were based on pipeline architectures where an automatic speech recognition (ASR) module generates a transcription of utterances and a SLU module predicts the semantic labels. Pipeline systems now tend to be replaced by end-to-end 1 architectures based on neural models, where semantic representations are produced directly from a speech input without using transcriptions [2,3,4,5]. Most of recently proposed end-to-end models are based on sequence-tosequence architectures.…”

Dialogue History Integration into End-to-End Signal-to-Concept Spoken Language Understanding Systems

“…Nowadays there is a growing research interest in end-to-end systems for various SLU tasks [23][24][25][26][27][28][29][30][31]. In this work, similarly to [26,29], end-to-end training of signal-to-concept models is performed through the recurrent neural network (RNN) architecture and the connectionist temporal classification (CTC) loss function [32] as shown in Figure 1.…”