End-to-End Multi-Modal Speech Recognition on an Air and Bone Conducted Speech Corpus

Wang, Mou; Chen, Junqi; Zhang, Xiao-Lei; Rahardja, Susanto

doi:10.1109/taslp.2022.3224305

Cited by 14 publications

(2 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to validate the scalability of RP based embeddings for speaker recognition on larger datasets, a standalone study was conducted on the publicly available ABCS corpus comprising synchronously recorded air and bone microphone speech from 100 (50 male and 50 female) speakers 72 . Random, 30 clean speech utterances from each of the 100 speakers for both air and bone conduction were chosen as the dataset.…”

Section: Resultsmentioning

confidence: 99%

Recurrence plot embeddings as short segment nonlinear features for multimodal speaker identification using air, bone and throat microphones

Nawas,

Shahina,

Balachandar

et al. 2024

Sci Rep

View full text Add to dashboard Cite

Speech is produced by a nonlinear, dynamical Vocal Tract (VT) system, and is transmitted through multiple (air, bone and skin conduction) modes, as captured by the air, bone and throat microphones respectively. Speaker specific characteristics that capture this nonlinearity are rarely used as stand-alone features for speaker modeling, and at best have been used in tandem with well known linear spectral features to produce tangible results. This paper proposes Recurrent Plot (RP) embeddings as stand-alone, non-linear speaker-discriminating features. Two datasets, the continuous multimodal TIMIT speech corpus and the consonant-vowel unimodal syllable dataset, are used in this study for conducting closed-set speaker identification experiments. Experiments with unimodal speaker recognition systems show that RP embeddings capture the nonlinear dynamics of the VT system which are unique to every speaker, in all the modes of speech. The Air (A), Bone (B) and Throat (T) microphone systems, trained purely on RP embeddings perform with an accuracy of 95.81%, 98.18% and 99.74%, respectively. Experiments using the joint feature space of combined RP embeddings for bimodal (A–T, A–B, B–T) and trimodal (A–B–T) systems show that the best trimodal system (99.84% accuracy) performs on par with trimodal systems using spectrogram (99.45%) and MFCC (99.98%). The 98.84% performance of the B–T bimodal system shows the efficacy of a speaker recognition system based entirely on alternate (bone and throat) speech, in the absence of the standard (air) speech. The results underscore the significance of the RP embedding, as a nonlinear feature representation of the dynamical VT system that can act independently for speaker recognition. It is envisaged that speech recognition too will benefit from this nonlinear feature.

show abstract

Section: Resultsmentioning

confidence: 99%

Recurrence plot embeddings as short segment nonlinear features for multimodal speaker identification using air, bone and throat microphones

Nawas,

Shahina,

Balachandar

et al. 2024

Sci Rep

View full text Add to dashboard Cite

show abstract

“…These systems harness the power of distributed computing and intelligent decision-making algorithms to tackle complex tasks and problems that may exceed the capacity of individual agents or centralized systems [8]. MIDS leverage collaborative efforts among diverse agents, each possessing specialized knowledge or skills, to achieve goals efficiently and effectively [9]. By distributing intelligence across the network, MIDS offer scalability, fault tolerance, and adaptability to dynamic environments.…”

Section: Iintroductionmentioning

confidence: 99%

Analysis of Multi-Intelligent Distributed Japanese Language Block Recognition Based on Knowledge Recognition Corpus

Huang

2024

jes

View full text Add to dashboard Cite

The Knowledge Recognition Corpus (KRC) serves as a comprehensive repository designed to capture and organize diverse forms of knowledge for computational analysis and understanding. This corpus encompasses a wide array of data sources, including text documents, images, videos, and structured datasets, covering various domains and topics. The concept of Multi-Intelligent Distributed Knowledge Recognition Corpus (MI-DKRC) represents an innovative approach to harnessing the collective intelligence of distributed systems for knowledge recognition tasks. This paper presents an analysis of Multi-Intelligent Distributed Japanese Language Block Recognition (MI-DJLBR) based on the Knowledge Recognition Corpus (KRC), employing multi-factor bi-gram Sentimental Classification (MFbi-SC). The MI-DJLBR system is designed to recognize and classify Japanese language blocks within a distributed framework, leveraging the collective intelligence of multiple intelligent agents. Through the utilization of the KRC, which encompasses a diverse array of Japanese text samples, images, and multimedia content, MI-DJLBR aims to enhance the efficiency and accuracy of Japanese language block recognition tasks. The incorporation of MFbi-SC further refines this process by considering multiple factors, including syntax, semantics, context, and sentiment, to classify Japanese language blocks with greater precision. Simulation demonstrated that a dataset of 1,000 Japanese language blocks, MI-DJLBR achieves an average recognition accuracy of 94.5%, demonstrating its effectiveness in accurately identifying and classifying Japanese text segments. Furthermore, the incorporation of MFbi-SC enhances the system's classification accuracy by an average of 7.2%, indicating the significance of multi-factor sentiment analysis in refining classification outcomes. In terms of computational efficiency, MI-DJLBR exhibits impressive processing times, with an average recognition speed of 200 blocks per second. This highlights the scalability and responsiveness of the distributed framework, enabling efficient processing of large-scale Japanese language datasets in real-time scenarios.

show abstract

Robust Perceptual Wavelet Packet Features for the Recognition of Spontaneous Kannada Sentences

Shanthamallappa,

Ravi

2023

Wireless Pers Commun

View full text Add to dashboard Cite

End-to-End Multi-Modal Speech Recognition on an Air and Bone Conducted Speech Corpus

Cited by 14 publications

References 34 publications

Recurrence plot embeddings as short segment nonlinear features for multimodal speaker identification using air, bone and throat microphones

Recurrence plot embeddings as short segment nonlinear features for multimodal speaker identification using air, bone and throat microphones

Analysis of Multi-Intelligent Distributed Japanese Language Block Recognition Based on Knowledge Recognition Corpus

Robust Perceptual Wavelet Packet Features for the Recognition of Spontaneous Kannada Sentences

Contact Info

Product

Resources

About