ASR Error Management for Improving Spoken Language Understanding

Simonnet, Edwin; Ghannay, Sahar; Camelin, Nathalie; Estève, Yannick; Mori, Renato De

doi:10.21437/interspeech.2017-1178

Cited by 30 publications

(37 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We think the starred mode must not be applied too early in the training chain, and would be applied on sub-tasks very close to the final target. In comparison to the literature, 16.4% of CER and 20.9% of CVER are very good results, since the best results ever published on the MEDIA test, when analysing speech instead of manual transcriptions, were a CER of 19.9% and a CVER of 25.1% [8]. For fair comparison with a state-of-the-art approach, we present in the next section a pipeline system we develop that takes benefits of the very good quality of our ASR outputs.…”

Section: Performance Of Curriculum-based Transfer Learningmentioning

confidence: 62%

“…For our experiments, we feed them with outputs from our end-to-end ASR system (with LM rescoring) fine tuned one the MEDIA training data. The word error rate (WER) of this system on the MEDIA data is 9.3%, that is very good in comparison to the 23.6% of WER got by the ASR used to reach the best CER/CVER values in the literacy until now [8]. Comparisons in CER/CVER values between state-of-the art pipeline approach and end-to-end system are provided in table 4.…”

Section: Performance Of Curriculum-based Transfer Learningmentioning

confidence: 79%

See 1 more Smart Citation

Curriculum-Based Transfer Learning for an Effective End-to-End Spoken Language Understanding and Domain Portability

Caubrière¹,

Tomashenko²,

Laurent³

et al. 2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

We present an end-to-end approach to extract semantic concepts directly from the speech audio signal. To overcome the lack of data available for this spoken language understanding approach, we investigate the use of a transfer learning strategy based on the principles of curriculum learning. This approach allows us to exploit out-of-domain data that can help to prepare a fully neural architecture. Experiments are carried out on the French MEDIA and PORTMEDIA corpora and show that this end-toend SLU approach reaches the best results ever published on this task. We compare our approach to a classical pipeline approach that uses ASR, POS tagging, lemmatizer, chunker... and other NLP tools that aim to enrich ASR outputs that feed an SLU text to concepts system. Last, we explore the promising capacity of our end-to-end SLU approach to address the problem of domain portability.

show abstract

Section: Performance Of Curriculum-based Transfer Learningmentioning

confidence: 62%

Section: Performance Of Curriculum-based Transfer Learningmentioning

confidence: 79%

Curriculum-Based Transfer Learning for an Effective End-to-End Spoken Language Understanding and Domain Portability

Caubrière¹,

Tomashenko²,

Laurent³

et al. 2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Both sets are described in section 4. Similar to [6] we report NLU performances on the VocADom@A4H test set using the concept error rate (CER) for slot labels. Intent classification is evaluated using the F1-score.…”

Section: Pipeline Slu Baseline Approachmentioning

confidence: 94%

“…More recently, acoustic word embeddings for ASR error detection were trained through a convolutional neural network (CNN) based ASR model to detect erroneous words [6]. Output of this ASR model was fed to a conditional random fields (CRF) model and an attention-based RNN NLU model.…”

Section: Related Workmentioning

confidence: 99%

SLU for Voice Command in Smart Home: Comparison of Pipeline and End-to-End Approaches

Desot

Portet

Vacher

2019

2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

View full text Add to dashboard Cite

HAL is a multidisciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

show abstract

“…More recently, to improve ASR error handling, acoustic word embeddings for ASR error detection were trained through a convolutional neural network (CNN) based ASR model to detect erroneous words. Output of this ASR model is fed to conditional random fields (CRF) and an attentionbased RNN NLU model [6]. The CRF outperformed the RNN approach and the concept error rate (CER) decreased by 1% integrating confidence measures.…”

Section: Related Workmentioning

confidence: 99%

Towards End-to-End spoken intent recognition in smart home

Desot

Portet

Vacher

2019

2019 International Conference on Speech Technology and Human-Computer Dialogue (SpeD)

View full text Add to dashboard Cite

on clean transcriptions whereas ASR transcriptions contain errors reducing the overall performance. Although the pipeline approach is widely adopted, there is a rising interest for end-toend (E2E) SLU which combines ASR and NLU in one model, avoiding the cumulative ASR and NLU errors of the pipeline approach [2], [3]. The main motivation for applying the E2E approach is that word by word recognition is not needed to infer intents. On top of that, the phoneme dictionary and language model (LM) of the ASR become optional. However, E2E approaches are highly dependent on large training data sets which are difficult to acquire, limiting the applicability to new domains where data is scarce which is the case for smart homes.The main contributions of this paper are: 1) the first work on E2E SLU for voice command in a smart home environment; 2) a comparison of a state-of-the-art pipeline approach that predicts intents from the ASR hypothesis and an E2E SLU model; 3) experiments performed with realistic non-English and synthetic data to deal with the paucity of domain specific data sets. Both approaches are positioned with respect to the state-of-the-art in Section II and are outlined in Section III. We tackle the lack of domain-specific data by using Natural Language Generation (NLG) and text-to-speech (TTS) to generate French voice command training data. An overview of these processes and data sets is given in Sections III and IV. Section V presents the results of experiments on a corpus of real smart home voice commands followed by a discussion, conclusion and outlook on future work. II. RELATED WORKSLU is typically seen as a slot-filling task in order to predict the speaker's intent on the one side and entities in a spoken utterance (slots and values) on the other side [1]. The most common approach is a pipeline of an ASR and an NLU module. The ASR system outputs the hypothesis transcriptions from a speech utterance that are analyzed by the NLU module to extract the meaning. While the slot-filling task is most often Abstract-Voice based interaction in a smart home has become a feature of many industrial products. These systems react to voice commands, whether it is for answering a question, providing music or turning on the lights. To be efficient, these systems must be able to extract the intent of the user from the voice command. Intent recognition from voice is typically performed through automatic speech recognition (ASR) and intent classification from the transcriptions in a pipeline. However, the errors accumulated at the ASR stage might severely impact the intent classifier. I n t his p aper, w e p ropose a n End-to-End (E2E) model to perform intent classification directly from the raw speech input. The E2E approach is thus optimized for this specific task and avoids error propagation. Furthermore, prosodic aspects of the speech signal can be exploited by the E2E model for intent classification (e.g., question vs imperative voice). Experiments on a corpus of voice commands acquired in a real smart home reveal t...

show abstract

ASR Error Management for Improving Spoken Language Understanding

Cited by 30 publications

References 18 publications

Curriculum-Based Transfer Learning for an Effective End-to-End Spoken Language Understanding and Domain Portability

Curriculum-Based Transfer Learning for an Effective End-to-End Spoken Language Understanding and Domain Portability

SLU for Voice Command in Smart Home: Comparison of Pipeline and End-to-End Approaches

Towards End-to-End spoken intent recognition in smart home

Contact Info

Product

Resources

About