A Study for Improving Device-Directed Speech Detection Toward Frictionless Human-Machine Interaction

Huang, Chien‐Chang; Maas, Roland; Mallidi, Sri Harish; Hoffmeister, Björn

doi:10.21437/interspeech.2019-2840

Cited by 21 publications

(35 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their model can achieve an EER (Equal Error Rate) of 5.2% on their proprietary dataset. In a study to improve device-directed utterance classification by [6], they present the performance for different dialog types and decoder features. Learning when to listen by [7] proposes using Prosodic features along with lexical features for better detection of utterance; they report EER of 6.72 with their best model.…”

Section: Related Workmentioning

confidence: 99%

On-Device System for Device Directed Speech Detection for Improving Human Computer Interaction

et al. 2021

View full text Add to dashboard Cite

Voice assistants (VA) are finding a place in many households, with increasing numbers. Nevertheless, for every interaction user invokes VA using a key or wake-up word, which is too common and hinders natural conversation. To solve this, we propose an On-Device solution that listens to the user continuously for only predefined period and classifies the utterance into device-directed or non-devicedirected using a deep learning-based model. Since our solution is On-Device, the privacy of the user is maintained. We tried to solve false acceptance of utterance as a command and incorrect rejection of the command as background noise to improve user experience. We calculated the False Acceptance Rate (FAR) and False Rejection Rate (FRR) for different thresholds to evaluate our model. Then we calculated Equal Error Rate (EER) results with the FRR and FAR data. We achieved 3.6% of EER on our best optimized Stage-1 model and EER of 5.1% on Stage-2 model. We have also used accuracy as evaluation metrics during training and testing. We achieved an accuracy of 96.8% on testing data with our Stage-1 model and 95.3% on Stage-2 model.

show abstract

Section: Related Workmentioning

confidence: 99%

On-Device System for Device Directed Speech Detection for Improving Human Computer Interaction

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Utterance-level acoustic embeddings computed using LSTMs have been previously used in [4,5,6] where they were combined with ASR decoding features for device-directed audio detection. Our approach depicted in Figure 1 differs from prior literature in multiple ways though.…”

Section: Introductionmentioning

confidence: 99%

Knowledge Transfer for Efficient on-Device False Trigger Mitigation

Dighe

Marchi

Vishnubhotla

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In this paper, we address the task of determining whether a given utterance is directed towards a voice-enabled smart-assistant device or not. An undirected utterance is termed as a "false trigger" and false trigger mitigation (FTM) is essential for designing a privacycentric non-intrusive smart assistant. The directedness of an utterance can be identified by running automatic speech recognition (ASR) and determining the user intent by analyzing the ASR transcript. Yet, in case of a false trigger, transcribing the audio using ASR itself is strongly undesirable. To alleviate this issue, we propose an LSTM-based FTM architecture which determines the user intent from acoustic features directly without explicitly generating ASR transcripts from the audio. The proposed models are smallfootprint and can be run on-device with limited computational resources. During training, the model parameters are optimized using a knowledge transfer approach where a more accurate self-attention graph neural network model [1] serves as the teacher. Given the whole audio snippets, our approach mitigates 87% of false triggers at 99% true positive rate (TPR), and in a streaming audio scenario, the system listens to only 1.69s of the false trigger audio before rejecting it while achieving the same TPR.

show abstract

“…Approaches for the classification of utterances into systemand non-system-directed ones typically use acoustic features extracted from the speech signal, e.g., [1,2,3,4,5]. Previous works [1,6,7] also show that using an attention mechanism *Author performed research herein as part of an internship-partnership program between Mila and Nuance. combined with a BiLSTM network can improve classification performance.…”

Section: Introductionmentioning

confidence: 99%

“…In addition to using conventional acoustic features, e.g., MFCCs or log filter-bank energies, some works also incorporate other acoustic and non-acoustic features as more representative cues of system-directed speech. Such works include [4] where prosodic features are used, and [2,4,6] where lexical and semantic features are derived from ASR decoders.…”

Section: Introductionmentioning

confidence: 99%

Improving Identification of System-Directed Speech Utterances by Deep Learning of ASR-Based Word Embeddings and Confidence Metrics

Vilaysouk

Nour-Eldin

Connolly

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In this paper, we extend our previous work on the detection of system-directed speech utterances. This type of binary classification can be used by virtual assistants to create a more natural and fluid interaction between the system and the user. We explore two methods that both improve the Equal-Error-Rate (EER) performance of the previous model. The first exploits the supplementary information independently captured by ASR models through integrating ASR decoder-based features as additional inputs to the final classification stage of the model. This relatively improves EER performance by 13%. The second proposed method further integrates word embeddings into the architecture and, when combined with the first method, achieves a significant EER performance improvement of 48%, relative to that of the baseline.

show abstract

A Study for Improving Device-Directed Speech Detection Toward Frictionless Human-Machine Interaction

Cited by 21 publications

References 4 publications

On-Device System for Device Directed Speech Detection for Improving Human Computer Interaction

On-Device System for Device Directed Speech Detection for Improving Human Computer Interaction

Knowledge Transfer for Efficient on-Device False Trigger Mitigation

Improving Identification of System-Directed Speech Utterances by Deep Learning of ASR-Based Word Embeddings and Confidence Metrics

Contact Info

Product

Resources

About