Multi-view Representation Learning via Canonical Correlation Analysis for Dysarthric Speech Recognition

Kim, Myungjong; Cao, Beiming; Wang, Jun

doi:10.1007/978-3-030-00214-5_133

Cited by 4 publications

(6 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our approach presents a possibility in effectively modeling dysarthric speech (even low intelligible speech) in a speaker-independent way. Future directions include 1) a test of the CLSTM-RNN approach using a larger dataset collected from more subjects, 2) applying speaker adaptation/normalization techniques [27], and 3) using articulatory information [25,29].…”

Section: Discussionmentioning

confidence: 99%

Dysarthric Speech Recognition Using Convolutional LSTM Neural Network

Kim

Cao

An³

et al. 2018

Interspeech 2018

Self Cite

View full text Add to dashboard Cite

Dysarthria is a motor speech disorder that impedes the physical production of speech. Speech in patients with dysarthria is generally characterized by poor articulation, breathy voice, and monotonic intonation. Therefore, modeling the spectral and temporal characteristics of dysarthric speech is critical for better performance in dysarthric speech recognition. Convolutional long short-term memory recurrent neural networks (CLSTM-RNNs) have recently successfully been used in normal speech recognition, but have rarely been used in dysarthric speech recognition. We hypothesized CLSTM-RNNs have the potential to capture the distinct characteristics of dysarthric speech, taking advantage of convolutional neural networks (CNNs) for extracting effective local features and LSTM-RNNs for modeling temporal dependencies of the features. In this paper, we investigate the use of CLSTM-RNNs for dysarthric speech recognition. Experimental evaluation on a database collected from nine dysarthric patients showed that our approach provides substantial improvement over both standard CNN and LSTM-RNN based speech recognizers.

show abstract

Section: Discussionmentioning

confidence: 99%

Dysarthric Speech Recognition Using Convolutional LSTM Neural Network

Kim

Cao

An³

et al. 2018

Interspeech 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…The range of each class length in the MNIST training dataset is [100 : 100 : 600]. We randomly select [5,6,7,8] samples per class for the ORL training dataset. The FERET training dataset includes [3,4,5,6] samples for each class.…”

Section: A Experimental Settingmentioning

confidence: 99%

“…We randomly select [5,6,7,8] samples per class for the ORL training dataset. The FERET training dataset includes [3,4,5,6] samples for each class. For each class and each training size, the training samples are randomly selected and the remaining samples are applied as the test dataset.…”

Section: A Experimental Settingmentioning

confidence: 99%

“…The research exploring multi-view data is multi-view learning. Multi-view learning has been utilized in many applications, such as face recognition [2], object detection [3], image classification [4], speech recognition [5], etc.…”

Section: Introductionmentioning

confidence: 99%

“…Most existing multi-view classification methods [2], [3], [5] have obtained satisfactory performances through exploiting the correlation between different views. Most existing…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-View Classification via a Fast and Effective Multi-View Nearest-Subspace Classifier

Shu¹,

Zhang²,

Tang³

2019

IEEE Access

View full text Add to dashboard Cite

Multi-view data represented in multiple views contains more complementary information than a single view, whereby multi-view learning explores and utilizes the multi-view data. In general, most existing multi-view learning methods consider the correlation between multiple views. However, the relationship between classes and views which is also important in multi-view learning has never been involved in the existing works. In this paper, we propose a fast and effective multi-view nearest-subspace classifier (MV-NSC) by taking advantage of both the two relationships simultaneously. MV-NSC consists of four main parts: 1) projection residual, 2) view-dependent class separability, 3) view similarity, and 4) final decision. The last part combines the first three parts in one final decision matrix, while the first three parts utilize the information of the multi-view data in various aspects. Our proposed method is evaluated on four benchmark datasets and compared with seven other classifiers including both multi-and single-view algorithms. According to the experimental results, it shows that our proposed method is effective, efficient, and robust in multi-view classification.INDEX TERMS Adaption model, classification algorithm, multi-view data, nearest-subspace classifier, view-dependent class separability, view similarity.

show abstract

MagTrack: A Wearable Tongue Motion Tracking System for Silent Speech Interfaces

Cao

Ravi

Sebkhi

et al. 2023

J Speech Lang Hear Res

View full text Add to dashboard Cite

Purpose: Current electromagnetic tongue tracking devices are not amenable for daily use and thus not suitable for silent speech interface and other applications. We have recently developed MagTrack, a novel wearable electromagnetic articulograph tongue tracking device. This study aimed to validate MagTrack for potential silent speech interface applications. Method: We conducted two experiments: (a) classification of eight isolated vowels in consonant–vowel–consonant form and (b) continuous silent speech recognition. In these experiments, we used data from healthy adult speakers collected with MagTrack. The performance of vowel classification was measured by accuracies. The continuous silent speech recognition was measured by phoneme error rates. The performance was then compared with results using data collected with commercial electromagnetic articulograph in a prior study. Results: The isolated vowel classification using MagTrack achieved an average accuracy of 89.74% when leveraging all MagTrack signals ( x , y , z coordinates; orientation; and magnetic signals), which outperformed the accuracy using commercial electromagnetic articulograph data (only y , z coordinates) in our previous study. The continuous speech recognition from two subjects using MagTrack achieved phoneme error rates of 73.92% and 66.73%, respectively. The commercial electromagnetic articulograph achieved 64.53% from the same subject (66.73% using MagTrack data). Conclusions: MagTrack showed comparable results with the commercial electromagnetic articulograph when using the same localized information. Adding raw magnetic signals would improve the performance of MagTrack. Our preliminary testing demonstrated the potential for silent speech interface as a lightweight wearable device. This work also lays the foundation to support MagTrack's potential for other applications including visual feedback–based speech therapy and second language learning.

show abstract

Multi-view Representation Learning via Canonical Correlation Analysis for Dysarthric Speech Recognition

Cited by 4 publications

References 25 publications

Dysarthric Speech Recognition Using Convolutional LSTM Neural Network

Dysarthric Speech Recognition Using Convolutional LSTM Neural Network

Multi-View Classification via a Fast and Effective Multi-View Nearest-Subspace Classifier

MagTrack: A Wearable Tongue Motion Tracking System for Silent Speech Interfaces

Contact Info

Product

Resources

About