Improved External Speaker-Robust Keyword Spotting for Hearing Assistive Devices

López-Espejo, Iván

doi:10.1109/taslp.2020.2984089

Cited by 17 publications

(9 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Personalized devices, such as hearing assistive devices, require the ability to detect external speakers and prevent them from triggering the device. [22] developed a multi-tasking keyword spotting model with the ability to detect non-users.…”

Section: Multi-tasking Models Attained By Joint Trainingmentioning

confidence: 99%

“…Joint training of speech and speaker recognition as a unified multi-tasking model usually has the following one or both benefits over training two independent models. First, the two tasks can share the data processing or feature learning pipeline to some extent [26,22,28,18,17]. Second, each can benefit from the improved performance of the other [28,18].…”

Section: Multi-tasking Models Attained By Joint Trainingmentioning

confidence: 99%

“…The two downstream tasks, keyword spotting and speaker verification, in [17] share the same wav2vec v2 backbone. The keyword spotting and own voice/external speaker detection tasks in [22] share the same residual deep learning network for feature extraction. [28] developed a multi-tasking model for speech and speaker recognition through collaborative training.…”

Section: Multi-tasking Models Attained By Joint Trainingmentioning

confidence: 99%

See 2 more Smart Citations

A Multi-tasking Model of Speaker-Keyword Classification for Keeping Human in the Loop of Drone-assisted Inspection

Liu¹,

Parsan²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Audio commands are a preferred communication medium to keep inspectors in the loop of civil infrastructure inspection performed by a semi-autonomous drone. To understand job-specific commands from a group of heterogeneous and dynamic inspectors, a model needs to be developed cost-effectively for the group and easily adapted when the group changes. This paper is motivated to build a multi-tasking deep learning model that possesses a Share-Split-Collaborate architecture. This architecture allows the two classification tasks to share the feature extractor and then split subject-specific and keyword-specific features intertwined in the extracted features through feature projection and collaborative training. A base model for a group of five authorized subjects is trained and tested on the inspection keyword dataset collected by this study. The model achieved a 95.3% or higher mean accuracy in classifying the keywords of any authorized inspectors. Its mean accuracy in speaker classification is 99.2%. Due to the richer keyword representations that the model learns from the pooled training data, adapting the base model to a new inspector requires only a little training data from that inspector, like five utterances per keyword. Using the speaker classification scores for inspector verification can achieve a success rate of at least 93.9% in verifying authorized inspectors and 76.1% in detecting unauthorized ones. Further, the paper demonstrates the applicability of the proposed model to larger-size groups on a public dataset. This paper provides a solution to addressing challenges facing AI-assisted human-robot interaction, including worker heterogeneity, worker dynamics, and job heterogeneity.

show abstract

Section: Multi-tasking Models Attained By Joint Trainingmentioning

confidence: 99%

Section: Multi-tasking Models Attained By Joint Trainingmentioning

confidence: 99%

Section: Multi-tasking Models Attained By Joint Trainingmentioning

confidence: 99%

See 1 more Smart Citation

A Multi-tasking Model of Speaker-Keyword Classification for Keeping Human in the Loop of Drone-assisted Inspection

Liu¹,

Parsan²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In this mode, the input segment x is often assigned to the class with highest posterior probability. The non-streaming deep KWS systems generated sharply peaked posterior distributions in [53][54]. One possible explanation for this phenomenon is that the non-streaming systems handle isolated, welldefined class realizations, and not the inter-class transition information, as opposed to the streaming system case.…”

Section: Introductionmentioning

confidence: 97%

Deep Learning-Based Detection of Inappropriate Speech Content for Film Censorship

et al. 2022

View full text Add to dashboard Cite

Audible content has become an effective tool for shaping one's personality and character due to the ease of accessibility to a huge audible content that could be an independent audio files or an audio of online videos, movies, and television programs. There is a huge necessity to filter inappropriate audible content of the easily accessible videos and films that are likely to contain an inappropriate speech content. With this in view, all the broadcasting and online video/audio platform companies hire a lot of manpower to detect the foul voices prior to censorship. The process has a large cost in terms of manpower, time and financial resources. In addition to inaccurate detection of foul voices due to fatigue of manpower and weakness of human visual and hearing system in long time and monotonous tasks. As such, this paper proposes an intelligent deep learning-based system for film censorship through a fast and accurate detection and localization approach using advanced deep Convolutional Neural Networks (CNNs). The dataset of foul language containing isolated words samples and continuous speech were collected, annotated, processed, and analyzed for the development of automated detection of inappropriate speech content. The results indicated the feasibility of the suggested systems by reporting a high volume of inappropriate spoken terms detection. The proposed system outperformed state-of-the-art baseline algorithms on the novel foul language dataset evaluation metrics in terms of macro average AUC (93.85%), weighted average AUC (94.58%), and all other metrics such as F1-score. Additionally, proposed acoustic system outperformed ASR-based system for profanity detection based on the evaluation metrics including AUC, accuracy, precision, and F1-score. Additionally, proposed system was proven to be faster than human manual screening and detection of audible content for films' censorship.

show abstract

“…Wake word spotting (WWS) can be considered as a specific case of keyword spotting (KWS), concerning the identification of predefined wake word(s) in utterances, often used for the wake-up of speech-enabled devices, such as "Hey Siri" in iPhone, "Alexa" in Amazon Echo, and "Ok Google" in Google Home [1,2,3,4], etc. In order to activate the interactions between devices and users, a standby wake word detection module is particularly important [5].…”

Section: Introductionmentioning

confidence: 99%

A Study of Designing Compact Audio-Visual Wake Word Spotting System Based on Iterative Fine-Tuning in Neural Network Pruning

Zhou¹,

Du²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Audio-only based wake word spotting (WWS) is challenging under noisy conditions due to the environmental interference in signal transmission. In this paper, we investigate on designing a compact audio-visual WWS system by utilizing the visual information to alleviate the degradation. Specifically, in order to use visual information, we first encode the detected lips to fixed-size vectors with Mo-bileNet and concatenate them with acoustic features followed by the fusion network for WWS. However, the audio-visual model based on neural network requires a large footprint and a high computational complexity. To meet the application requirements, we introduce neural network pruning strategy via the lottery ticket hypothesis in an iterative fine-tuning manner (LTH-IF), to the single-modal and multi-modal models, respectively. Tested on our in-house corpus for audio-visual WWS in a home TV scene, the proposed audiovisual system achieves significant performance improvements over the single-modality (audio-only or video-only) system under different noisy conditions. Moreover, LTH-IF pruning can largely reduce the network parameters and computations with no degradation of WWS performance, leading to a potential product solution for the TV wake-up scenario.

show abstract

Improved External Speaker-Robust Keyword Spotting for Hearing Assistive Devices

Cited by 17 publications

References 38 publications

A Multi-tasking Model of Speaker-Keyword Classification for Keeping Human in the Loop of Drone-assisted Inspection

A Multi-tasking Model of Speaker-Keyword Classification for Keeping Human in the Loop of Drone-assisted Inspection

Deep Learning-Based Detection of Inappropriate Speech Content for Film Censorship

A Study of Designing Compact Audio-Visual Wake Word Spotting System Based on Iterative Fine-Tuning in Neural Network Pruning

Contact Info

Product

Resources

About