End-to-End Named Entity Recognition from English Speech

Yadav, Hemant; Ghosh, Sreyan; Yu, Yi; Shah, Rajiv Ratn

doi:10.21437/interspeech.2020-2482

Cited by 29 publications

(29 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar to the O2O model trained with the conditional chain mapping in Section 2.2.2, this framework does not assume the conditional independence between output labels and has the flexibility to model the dependency between words/morphemes and linguistic annotations. Related works are using the O2O model, e.g., (Yadav et al, 2020), but they are based on CTC and do not consider such an explicit output dependency. Also, the proposed method using Transformer can preserve a relationship between the word/morpheme and the corresponding linguistic annotations across the sequence based on the aligned representation s i in Eq.…”

Section: O2o Model Trained With Conditional Chain Mappingmentioning

confidence: 99%

“…We investigated three models: selfattention-based CTC (Pham et al, 2019), the Transformer (Dong et al, 2018), and a hybrid Transformer trained with an auxiliary CTC objective (Transformer+CTC) (Karita et al, 2019). The CTC model was used in prior studies based on O2O models, e.g., (Audhkhasi et al, 2018;Yadav et al, 2020). During training, the CTC model was regularized with the Transformer decoder in the multitask learning fashion similar to Transformer+CTC.…”

Section: E2e Asrmentioning

confidence: 99%

“…1 shows a diagram of these systems. These approaches use one of the following models: a one-to-many (O2M) model (Kubo and Bacchiani, 2020;Ueno et al, 2018;Gowda et al, 2019;Sanabria and Metze, 2018;Adams et al, 2019), a one-to-one (O2O) model with a conditional chain mapping (Shi et al, 2020), or an O2O model with a single sequence (Audhkhasi et al, 2018;Ghannay et al, 2018;Shafey et al, 2019;Yadav et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

End-to-end ASR to jointly predict transcriptions and linguistic annotations

Omachi¹,

Fujita²,

Watanabe³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

We propose a Transformer-based sequence-tosequence model for automatic speech recognition (ASR) capable of simultaneously transcribing and annotating audio with linguistic information such as phonemic transcripts or part-of-speech (POS) tags. Since linguistic information is important in natural language processing (NLP), the proposed ASR is especially useful for speech interface applications, including spoken dialogue systems and speech translation, which combine ASR and NLP. To produce linguistic annotations, we train the ASR system using modified training targets: each grapheme or multi-grapheme unit in the target transcript is followed by an aligned phoneme sequence and/or POS tag. Since our method has access to the underlying audio data, we can estimate linguistic annotations more accurately than pipeline approaches in which NLP-based methods are applied to a hypothesized ASR transcript. Experimental results on Japanese and English datasets show that the proposed ASR system is capable of simultaneously producing highquality transcriptions and linguistic annotations.

show abstract

Section: O2o Model Trained With Conditional Chain Mappingmentioning

confidence: 99%

Section: E2e Asrmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

End-to-end ASR to jointly predict transcriptions and linguistic annotations

Omachi¹,

Fujita²,

Watanabe³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

show abstract

“…While entity extraction from text is well researched in the literature, NER on speech is less studied. Most initial works on speech had a two staged approach -ASR followed by NER (Cohn et al, 2019), recent works directly extract entities from speech (Ghannay et al, 2018;Yadav et al, 2020). While NER helps in ecommerce search on websites and apps, the specific nature of order identification problem and the limited search space of active orders make NER unnecessary.…”

Section: Related Workmentioning

confidence: 99%

“…Additionally, NER systems' performance are comparatively poor on ASR transcripts owing to high degree of recognition and lexical noise (e.g. missing capitalization etc) (Yadav et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

“Are you calling for the vaporizer you ordered?” Combining Search and Prediction to Identify Orders in Contact Centers

.K¹,

Roy²

2021

Proceedings of the 4th Workshop on E-Commerce and NLP

View full text Add to dashboard Cite

With the growing footprint of ecommerce worldwide, the role of contact center is becoming increasingly crucial for customer satisfaction. To effectively handle scale and manage operational cost, automation through chatbots and voice-bots are getting rapidly adopted. With customers having multiple, often long list of active orders -the first task of a voicebot is to identify which one they are calling about. Towards solving this problem which we refer to as order identification, we propose a two-staged real-time technique by combining search and prediction in a sequential manner. In the first stage, analogous to retrieval-based question-answering, a fuzzy search technique uses customized lexical and phonetic similarity measures on noisy transcripts of calls to retrieve the order of interest. The coverage of fuzzy search is limited by no or limited response from customers to voice prompts. Hence, in the second stage, a predictive solution that predicts the most likely order a customer is calling about based on certain features of orders is introduced. We compare with multiple relevant techniques based on word embeddings as well as ecommerce product search to show that the proposed approach provides the best performance with 64% coverage and 87% accuracy on a large real-life data-set. A system based on the proposed technique is also deployed in production for a fraction of calls landing in the contact center of a large ecommerce provider; providing real evidence of operational benefits as well as increased customer delight.

show abstract

Attention-Based End-to-End Named Entity Recognition from Speech

Porjazovski

Leinonen

Kurimo

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Named entities are heavily used in the field of spoken language understanding, which uses speech as an input. The standard way of doing named entity recognition from speech involves a pipeline of two systems, where first the automatic speech recognition system generates the transcripts, and then the named entity recognition system produces the named entity tags from the transcripts. In such cases, automatic speech recognition and named entity recognition systems are trained independently, resulting in the automatic speech recognition branch not being optimized for named entity recognition and vice versa. In this paper, we propose two attention-based approaches for extracting named entities from speech in an end-to-end manner, that show promising results. We compare both attention-based approaches on Finnish, Swedish, and English data sets, underlining their strengths and weaknesses.

show abstract

End-to-End Named Entity Recognition from English Speech

Cited by 29 publications

References 16 publications

End-to-end ASR to jointly predict transcriptions and linguistic annotations

End-to-end ASR to jointly predict transcriptions and linguistic annotations

“Are you calling for the vaporizer you ordered?” Combining Search and Prediction to Identify Orders in Contact Centers

Attention-Based End-to-End Named Entity Recognition from Speech

Contact Info

Product

Resources

About