End-to-End Architectures for ASR-Free Spoken Language Understanding

Palogiannidi, Elisavet; Gkinis, Ioannis; Mastrapas, George; Mizera, Petr; Stafylakis, Themos

doi:10.1109/icassp40776.2020.9054314

Cited by 21 publications

(14 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[22] has a comparable performance to ours (1.9% vs 2.18%) similar to the work of Palogiannidi et al [23], which has performances between 1.38% and 5.83% depending on the number and the types of the recurrent layers. Our approach is more appropriate for low-power devices since it is built from only convolutional layers, where the models of [6,22,23] includes many recurrent layers, which are slower compared to convolutional layers [9][10][11]. Figure 2 shows the validation losses for the three models trained respectively, with global CMVN, utterance CMVN and without any CMVN (No-CMVN).…”

Section: The Proposed Network Architecturesupporting

confidence: 89%

A Low Latency ASR-Free End to End Spoken Language Understanding System

Mhiri

Myer²,

Tomar

2020

Interspeech 2020

View full text Add to dashboard Cite

In recent years, developing a speech understanding system that classifies a waveform to structured data, such as intents and slots, without first transcribing the speech to text has emerged as an interesting research problem. This work proposes such as system with an additional constraint of designing a system that has a small enough footprint to run on small micro-controllers and embedded systems with minimal latency. Given a streaming input speech signal, the proposed system can process it segment-by-segment without the need to have the entire stream at the moment of processing. The proposed system is evaluated on the publicly available Fluent Speech Commands dataset. Experiments show that the proposed system yields state-of-theart performance with the advantage of low latency and a much smaller model when compared to other published works on the same task.

show abstract

Section: The Proposed Network Architecturesupporting

confidence: 89%

A Low Latency ASR-Free End to End Spoken Language Understanding System

Mhiri

Myer²,

Tomar

2020

Interspeech 2020

View full text Add to dashboard Cite

show abstract

“…SLU systems have traditionally been a cascade of an automatic speech recognition (ASR) system converting speech into text followed by a natural language understanding (NLU) system that interprets the meaning of the text [1][2][3][4]. In contrast, an end-to-end (E2E) SLU system [5][6][7][8][9][10][11][12][13][14] processes speech input directly into meaning without going through an intermediate text transcript.…”

Section: Introductionmentioning

confidence: 99%

Improving End-to-End Models for Set Prediction in Spoken Language Understanding

Kuo¹,

Tüske²,

Thomas³

et al. 2022

Preprint

View full text Add to dashboard Cite

The goal of spoken language understanding (SLU) systems is to determine the meaning of the input speech signal, unlike speech recognition which aims to produce verbatim transcripts. Advances in end-to-end (E2E) speech modeling have made it possible to train solely on semantic entities, which are far cheaper to collect than verbatim transcripts. We focus on this set prediction problem, where entity order is unspecified. Using two classes of E2E models, RNN transducers and attention based encoder-decoders, we show that these models work best when the training entity sequence is arranged in spoken order. To improve E2E SLU models when entity spoken order is unknown, we propose a novel data augmentation technique along with an implicit attention based alignment method to infer the spoken order. F1 scores significantly increased by more than 11% for RNN-T and about 2% for attention based encoder-decoder SLU models, outperforming previously reported results.

show abstract

“…Rather than containing discrete ASR and NLU modules, E2E SLU models are trained to infer the utterance semantics directly from the spoken signal [13][14][15][16][17][18][19][20]. These models are trained to maximize the SLU prediction accuracy where the predicted semantic targets vary from just the intent [21,22], to a full interpretation with domain, intents, and slots [13].…”

Section: Introductionmentioning

confidence: 99%

End-to-End Spoken Language Understanding for Generalized Voice Assistants

Saxon¹,

Choudhary²,

McKenna³

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

End-to-end (E2E) spoken language understanding (SLU) systems predict utterance semantics directly from speech using a single model. Previous work in this area has focused on targeted tasks in fixed domains, where the output semantic structure is assumed a priori and the input speech is of limited complexity. In this work we present our approach to developing an E2E model for generalized SLU in commercial voice assistants (VAs). We propose a fully differentiable, transformer-based, hierarchical system that can be pretrained at both the ASR and NLU levels. This is then fine-tuned on both transcription and semantic classification losses to handle a diverse set of intent and argument combinations. This leads to an SLU system that achieves significant improvements over baselines on a complex internal generalized VA dataset with a 43% improvement in accuracy, while still meeting the 99% accuracy benchmark on the popular Fluent Speech Commands dataset. We further evaluate our model on a hard test set, exclusively containing slot arguments unseen in training, and demonstrate a nearly 20% improvement, showing the efficacy of our approach in truly demanding VA scenarios.

show abstract

End-to-End Architectures for ASR-Free Spoken Language Understanding

Cited by 21 publications

References 19 publications

A Low Latency ASR-Free End to End Spoken Language Understanding System

A Low Latency ASR-Free End to End Spoken Language Understanding System

Improving End-to-End Models for Set Prediction in Spoken Language Understanding

End-to-End Spoken Language Understanding for Generalized Voice Assistants

Contact Info

Product

Resources

About