Incorporating speech recognition confidence into discriminative named entity recognition of speech data

Sudoh, Katsuhito; Tsukada, Hajime; Isozaki, Hideki

doi:10.3115/1220175.1220253

Cited by 26 publications

(18 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another method is to learn NLU models on noisy ASR transcriptions. In [5], manual and ASR output transcriptions with word ASR confidence measures were used for a NER task, to learn a support vector machine-based (SVM) NER system. This increased precision by 2% as compared to the baseline.…”

Section: Related Workmentioning

confidence: 99%

Towards End-to-End spoken intent recognition in smart home

Desot

Portet

Vacher

2019

2019 International Conference on Speech Technology and Human-Computer Dialogue (SpeD)

View full text Add to dashboard Cite

on clean transcriptions whereas ASR transcriptions contain errors reducing the overall performance. Although the pipeline approach is widely adopted, there is a rising interest for end-toend (E2E) SLU which combines ASR and NLU in one model, avoiding the cumulative ASR and NLU errors of the pipeline approach [2], [3]. The main motivation for applying the E2E approach is that word by word recognition is not needed to infer intents. On top of that, the phoneme dictionary and language model (LM) of the ASR become optional. However, E2E approaches are highly dependent on large training data sets which are difficult to acquire, limiting the applicability to new domains where data is scarce which is the case for smart homes.The main contributions of this paper are: 1) the first work on E2E SLU for voice command in a smart home environment; 2) a comparison of a state-of-the-art pipeline approach that predicts intents from the ASR hypothesis and an E2E SLU model; 3) experiments performed with realistic non-English and synthetic data to deal with the paucity of domain specific data sets. Both approaches are positioned with respect to the state-of-the-art in Section II and are outlined in Section III. We tackle the lack of domain-specific data by using Natural Language Generation (NLG) and text-to-speech (TTS) to generate French voice command training data. An overview of these processes and data sets is given in Sections III and IV. Section V presents the results of experiments on a corpus of real smart home voice commands followed by a discussion, conclusion and outlook on future work. II. RELATED WORKSLU is typically seen as a slot-filling task in order to predict the speaker's intent on the one side and entities in a spoken utterance (slots and values) on the other side [1]. The most common approach is a pipeline of an ASR and an NLU module. The ASR system outputs the hypothesis transcriptions from a speech utterance that are analyzed by the NLU module to extract the meaning. While the slot-filling task is most often Abstract-Voice based interaction in a smart home has become a feature of many industrial products. These systems react to voice commands, whether it is for answering a question, providing music or turning on the lights. To be efficient, these systems must be able to extract the intent of the user from the voice command. Intent recognition from voice is typically performed through automatic speech recognition (ASR) and intent classification from the transcriptions in a pipeline. However, the errors accumulated at the ASR stage might severely impact the intent classifier. I n t his p aper, w e p ropose a n End-to-End (E2E) model to perform intent classification directly from the raw speech input. The E2E approach is thus optimized for this specific task and avoids error propagation. Furthermore, prosodic aspects of the speech signal can be exploited by the E2E model for intent classification (e.g., question vs imperative voice). Experiments on a corpus of voice commands acquired in a real smart home reveal t...

show abstract

Section: Related Workmentioning

confidence: 99%

Towards End-to-End spoken intent recognition in smart home

Desot

Portet

Vacher

2019

2019 International Conference on Speech Technology and Human-Computer Dialogue (SpeD)

View full text Add to dashboard Cite

show abstract

“…Another method is to learn NLU models on noisy ASR output transcriptions. In [5], manual and ASR output transcriptions with word ASR confidence measures were used for a NER task, to learn a support vector machine-based (SVM) NE recognition system.…”

Section: Related Workmentioning

confidence: 99%

SLU for Voice Command in Smart Home: Comparison of Pipeline and End-to-End Approaches

Desot

Portet

Vacher

2019

2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

View full text Add to dashboard Cite

HAL is a multidisciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

show abstract

“…Early work on name error detection incorporated ASR word confidence estimates in a named entity recognition (NER) system [10,11,12], taking advantage of local contextual cues to names (e.g. titles for person names).…”

Section: Introductionmentioning

confidence: 99%

Effective data-driven feature learning for detecting name errors in automatic speech recognition

Marin

Ostendorf

2014

2014 IEEE Spoken Language Technology Workshop (SLT)

View full text Add to dashboard Cite

This paper addresses the problem of detecting name errors in automatic speech recognition (ASR) output. The highly skewed label distributions (i.e. name errors are infrequent), sparse training data, and large number of potential lexical features pose significant challenges for training name error classification systems. Data-driven feature learning is needed for handling multiple languages but is sensitive to over fitting. We address the problem by designing aggregate features using a related (sentence-level name detection) task, and reduce dimensionality of the lexical features using word classes. Experiments on conversational domain data in both English and Iraqi Arabic show that best results are obtained using all feature mapping methods plus feature selection using L1 regularization.

show abstract

Incorporating speech recognition confidence into discriminative named entity recognition of speech data

Cited by 26 publications

References 16 publications

Towards End-to-End spoken intent recognition in smart home

Towards End-to-End spoken intent recognition in smart home

SLU for Voice Command in Smart Home: Comparison of Pipeline and End-to-End Approaches

Effective data-driven feature learning for detecting name errors in automatic speech recognition

Contact Info

Product

Resources

About