Combining Continuous Word Representation and Prosodic Features for ASR Error Prediction

Ghannay, Sahar; Estève, Yannick; Camelin, Nathalie; Dutrey, Camille; Santiago, Fabián; Adda-Decker, Martine

doi:10.1007/978-3-319-25789-1_9

Cited by 12 publications

(17 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This hypothesis is evaluated by the softmax value of the Correct label scored with the MS-MLP. Experiments have shown that this is a calibrated confidence measure more effective than word posterior probability when comparison is based on the Normalized Cross Entropy (NCE) [15], which measures the information contribution provided by confidence knowledge. Table 1 shows the NCE values obtained by these two confidence measures on the MEDIA test data whose details can be found in section 5.…”

Section: Asr Error Detection and Confidence Measurementioning

confidence: 99%

ASR Error Management for Improving Spoken Language Understanding

Simonnet¹,

Ghannay²,

Camelin³

et al. 2017

Interspeech 2017

Self Cite

View full text Add to dashboard Cite

This paper addresses the problem of automatic speech recognition (ASR) error detection and their use for improving spoken language understanding (SLU) systems. In this study, the SLU task consists in automatically extracting, from ASR transcriptions, semantic concepts and concept/values pairs in a e.g touristic information system. An approach is proposed for enriching the set of semantic labels with error specific labels and by using a recently proposed neural approach based on word embeddings to compute well calibrated ASR confidence measures. Experimental results are reported showing that it is possible to decrease significantly the Concept/Value Error Rate with a state of the art system, outperforming previously published results performance on the same experimental data. It also shown that combining an SLU approach based on conditional random fields with a neural encoder/decoder attention based architecture, it is possible to effectively identifying confidence islands and uncertain semantic output segments useful for deciding appropriate error handling actions by the dialogue manager strategy.

show abstract

Section: Asr Error Detection and Confidence Measurementioning

confidence: 99%

ASR Error Management for Improving Spoken Language Understanding

Simonnet¹,

Ghannay²,

Camelin³

et al. 2017

Interspeech 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…Hence, they can capture different types of information: semantic, syntactic, etc. In our previous studies [5,11], we evaluated different kinds of word embeddings, including word2vecf on dependency trees [12], skip-gram provided by word2vec [13], and GloVe [14]. These evaluations were carried on ASR error detection, natural language processing, analogical and similarity tasks.…”

Section: Asr Error Detection Systemmentioning

confidence: 99%

“…We revealed that the combination of word embeddings through auto-encoder yields the best results compared to the other combination approaches (PCA and simple concatenation). Based on the results of these studies, we propose to use the best word embeddings (the three ones cited above) retained from the evaluation task [11] and to combine them with auto-encoder as in [5]. A detailed description of the word embeddings and the combination approaches is presented in [11,5].…”

Section: Asr Error Detection Systemmentioning

confidence: 99%

“…This approach is compared to Support Vector Machines, MaxEnt, and Extremely Randomized Tree classifiers, but not to CRFs. In [5], we proposed a neural approach to detect errors in automatic transcriptions, and to calibrate confidence measures provided by an ASR system. In this previous study, we experimented the use of several information sources: syntactic, lexical, ASR-based feature, however, we did not investigate the use of acoustic information, except to some prosodic features.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Acoustic Word Embeddings for ASR Error Detection

Ghannay¹,

Estève²,

Camelin³

et al. 2016

Interspeech 2016

Self Cite

View full text Add to dashboard Cite

This paper focuses on error detection in Automatic Speech Recognition (ASR) outputs. A neural network architecture is proposed, which is well suited to handle continuous word representations, like word embeddings. In a previous study, the authors explored the use of linguistic word embeddings, and more particularly their combination. In this new study, the use of acoustic word embeddings is explored. Acoustic word embeddings offer the opportunity of an a priori acoustic representation of words that can be compared, in terms of similarity, to an embedded representation of the audio signal. First, we propose an approach to evaluate the intrinsic performances of acoustic word embeddings in comparison to orthographic representations in order to capture discriminative phonetic information. Since French language is targeted in experiments, a particular focus is made on homophone words. Then, the use of acoustic word embeddings is evaluated for ASR error detection. The proposed approach gets a classification error rate of 7.94% while the previous state-of-the-art CRFbased approach gets a CER of 8.56% on the outputs of the ASR system which won the ETAPE evaluation campaign on speech recognition of French broadcast news.

show abstract

“…Dans ces travaux (Tam et al, 2014;Ogawa & Hori, 2017), différentes architectures neuronales ont été exploitées : perceptrons multi-couches, réseaux de neurones récurrents, etc. Dans nos études précédentes (Ghannay et al, 2015c(Ghannay et al, , 2016a, nous avons étudié l'utilisation de différents types d'embeddings de mot. Dans (Ghannay et al, 2015c), nous avons proposé une approche neuronale pour la détection d'erreurs dans les transcriptions automatiques et pour la calibration des mesures de confiance issues d'un SRAP.…”

Section: Introductionunclassified

Représentations de phrases dans un espace continu spécifiques à la tâche de détection d'erreurs

Ghannay¹,

Camelin²,

Estève³

2018

XXXIIe Journées D’Études Sur La Parole

Self Cite

View full text Add to dashboard Cite

Cet article présente une étude sur la modélisation des erreurs de reconnaissance de la parole au niveau de la phrase, afin de compenser certains phénomènes mis en avant par l'analyse des sorties du système de détection d'erreurs que nous avons précédemment proposé. Nous avons étudié trois approches différentes, qui sont fondées respectivement sur l'utilisation des représentations continues (embeddings) de phrases dédiées à la tâche de détection d'erreur, d'un modèle contextuel probabiliste (MCP) et d'un réseau de neurones récurrent BLSTM. Une approche pour construire les embeddings spécifiques à la tâche est proposée et comparée à l'approche Doc2vec. Les expériences sont effectuées sur des transcriptions automatiques du corpus ETAPE générées par le système de reconnaissance automatique du LIUM. Elles montrent que les embeddings spécifiques à la tâche obtiennent de meilleurs résultats que les embeddings génériques et que leur intégration dans notre système améliore les résultats par rapport aux MCP et BLSTM.

show abstract

Combining Continuous Word Representation and Prosodic Features for ASR Error Prediction

Cited by 12 publications

References 13 publications

ASR Error Management for Improving Spoken Language Understanding

ASR Error Management for Improving Spoken Language Understanding

Acoustic Word Embeddings for ASR Error Detection

Représentations de phrases dans un espace continu spécifiques à la tâche de détection d'erreurs

Contact Info

Product

Resources

About