End-To-End Named Entity And Semantic Concept Extraction From Speech

Abstract

Named entity recognition (NER) is among SLU tasks that usually extract semantic information from textual documents. Until now, NER from speech is made through a pipeline process that consists in processing first an automatic speech recognition (ASR) on the audio and then processing a NER on the ASR outputs. Such approach has some disadvantages (error propagation, metric to tune ASR systems sub-optimal in regards to the final task, reduced space search at the ASR output level,...) and it is known that more integrated approaches outperform sequential ones, when they can be applied. In this paper, we present a first study of end-to-end approach that directly extracts named entities from speech, though a unique neural architecture. On a such way, a joint optimization is able for both ASR and NER. Experiments are carried on French data easily accessible, composed of data distributed in several evaluation campaign. Experimental results show that this end-to-end approach provides better results (F-measure=0.69 on test data) than a classical pipeline approach to detect named entity categories (F-measure=0.65).

A Data Efficient End-to-End Spoken Language Understanding Architecture

“…We evaluate both ASR (Word Error Rate) and SLU (Concept Error Rate) results on MEDIA corpus (Dev and Test). State-of-the-art Models E2E SLU [4] 300h 30.1 27.0 E2E Baseline [3] 41.5h -39.8 E2E SLU [3] 500h -23.7 E2E SLU + curr. [3] 500h -16.4 ASR results are presented in Table 2.…”

“…4 SLU performances are given in Table 3. Our results can be compared with some previous works [4,3]. We note however that results reported in [4,3] are obtained with models trained with much more data exploiting NER tasks with transfer learning.…”

“…Using the example in Figure 1, the words "chambre double" (double room) for instance, instantiate the concept chambre-type. Our model, similar to [4], generates the output < chambre double chambretype >, which allows for attribute value extraction. 5 This choice constrains the model to learn chunking and tagging at the same time, which is a much harder problem than just tagging [1].…”

Dialogue History Integration into End-to-End Signal-to-Concept Spoken Language Understanding Systems

“…Nowadays there is a growing research interest in end-to-end systems for various SLU tasks [23][24][25][26][27][28][29][30][31]. In this work, similarly to [26,29], end-to-end training of signal-to-concept models is performed through the recurrent neural network (RNN) architecture and the connectionist temporal classification (CTC) loss function [32] as shown in Figure 1. A spectrogram of power normalized audio clips calculated on 20ms windows is used as the input features for the system.…”

“…First, we integrated dialog history into this system based on dialog history embedding vectors (h-vectors) as shown in Figure 1 and proposed in Section 3. Second, in this paper, the task is SF, therefore the output sequence besides the alphabetic characters also contains special characters corresponding to the semantic tags [26,29].…”