Constructing long short-term memory based deep recurrent neural networks for large vocabulary speech recognition

Li, Xiangang; Wu, Xihong

doi:10.1109/icassp.2015.7178826

Cited by 260 publications

(122 citation statements)

References 47 publications

Supporting

Mentioning

118

Contrasting

Unclassified

Order By: Relevance

“…Several works report improvements using CNNs, TDNNs and LSTMs with respect to DNNs, especially for large vocabularies [17,1,13,18,7,11]. We do not find this behaviour in our task.…”

Section: Resultscontrasting

confidence: 58%

Automatic Speech Recognition with Deep Neural Networks for Impaired Speech

España-Bonet

Fonollosa

2016

Advances in Speech and Language Technologies for Iberian Languages

View full text Add to dashboard Cite

Abstract. Automatic Speech Recognition has reached almost human performance in some controlled scenarios. However, recognition of impaired speech is a difficult task for two main reasons: data is (i) scarce and (ii) heterogeneous. In this work we train different architectures on a database of dysarthric speech. A comparison between architectures shows that, even with a small database, hybrid DNN-HMM models outperform classical GMM-HMM according to word error rate measures. A DNN is able to improve the recognition word error rate a 13% for subjects with dysarthria with respect to the best classical architecture. This improvement is higher than the one given by other deep neural networks such as CNNs, TDNNs and LSTMs. All the experiments have been done with the Kaldi toolkit for speech recognition for which we have adapted several recipes to deal with dysarthric speech and work on the TORGO database. These recipes are publicly available.

show abstract

“…Several works report improvements using CNNs, TDNNs and LSTMs with respect to DNNs, especially for large vocabularies [17,1,13,18,7,11]. We do not find this behaviour in our task.…”

Section: Resultscontrasting

confidence: 58%

Automatic Speech Recognition with Deep Neural Networks for Impaired Speech

España-Bonet

Fonollosa

2016

Advances in Speech and Language Technologies for Iberian Languages

View full text Add to dashboard Cite

show abstract

“…To understand class-specific spectral characteristics in the EEG recordings, we analyzed band powers in five frequency ranges: delta (0-4 Hz), theta (4-8 Hz), alpha (8)(9)(10)(11)(12)(13)(14), low beta (14)(15)(16)(17)(18)(19)(20), high beta (20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30) and low gamma .…”

Section: Visualizations Of the Spectral Differences Between Normalmentioning

confidence: 99%

Deep learning with convolutional neural networks for decoding and visualization of EEG pathology

Schirrmeister

Gemein

Eggensperger

et al. 2017

2017 IEEE Signal Processing in Medicine and Biology Symposium (SPMB)

145

View full text Add to dashboard Cite

Abstract-We apply convolutional neural networks (ConvNets) to the task of distinguishing pathological from normal EEG recordings in the Temple University Hospital EEG Abnormal Corpus. We use two basic, shallow and deep ConvNet architectures recently shown to decode task-related information from EEG at least as well as established algorithms designed for this purpose. In decoding EEG pathology, both ConvNets reached substantially better accuracies (about 6% better, ≈85% vs. ≈79%) than the only published result for this dataset, and were still better when using only 1 minute of each recording for training and only six seconds of each recording for testing. We used automated methods to optimize architectural hyperparameters and found intriguingly different ConvNet architectures, e.g., with max pooling as the only nonlinearity. Visualizations of the ConvNet decoding behavior showed that they used spectral power changes in the delta (0-4 Hz) and theta (4-8 Hz) frequency range, possibly alongside other features, consistent with expectations derived from spectral analysis of the EEG data and from the textual medical reports. Analysis of the textual medical reports also highlighted the potential for accuracy increases by integrating contextual information, such as the age of subjects. In summary, the ConvNets and visualization techniques used in this study constitute a next step towards clinically useful automated EEG diagnosis and establish a new baseline for future work on this topic.I. I Electroencephalography (EEG) is widely used in clinical practice because of its low cost and its lack of side effects due to its noninvasive nature. It is important both as a screening method as well as for hypothesis-based diagnostics, e.g., in epilepsy or stroke. One of the main limitations of using EEG for diagnostics is the required time and specialized knowledge of experts that need to be well-trained on EEG diagnostics to reach reliable results. Therefore, a machine-learning approach that aids in the diagnostic process could make EEG diagnosis more widely accessible, reduce time and effort for clinicians and potentially make diagnoses more accurate.In recent years researchers have increasingly addressed the field of computer-aided EEG diagnosis. So far, the applications were mostly limited to specific diagnoses such as Alzheimer's disease regression, neural networks, and more. This large variety of used methods indicates that the search for the best decoding approach for diverse types of EEG diagnosis is still ongoing.To overcome the lack of large datasets representative of the variety of EEG-diagnosable diseases and the heterogeneity of clinical populations, the Temple University Hospital (TUH) has published an unprecedented public dataset of clinical EEG recordings [6]. From this dataset with over 16000 clinical recordings, the TUH Abnormal EEG Corpus with about 3000 recordings has been created specifically to foster the development of methods for distinguishing pathological from normal EEG. Due to its size and rich annotatio...

show abstract

“…For example, deep ConvNets reduced the error rates on the ImageNet image‐recognition challenge, where 1.2 million images must be classified into 1000 different classes, from above 26% to below 4% within 4 years [He et al, ; Krizhevsky et al, ]. ConvNets also reduced error rates in recognizing speech, for example, from English news broadcasts [Sainath et al, ,; Sercu et al, ]; however, in this field, hybrid models combining ConvNets with other machine‐learning components, notably recurrent networks, and deep neural networks without convolutions are also competitive [Li and Wu, ; Sainath et al, ; Sak et al, ]. Deep ConvNets also contributed to the spectacular success of AlphaGo, an artificial intelligence that beat the world champion in the game of Go [Silver et al, ].…”

Section: Introductionmentioning

confidence: 99%

Deep learning with convolutional neural networks for EEG decoding and visualization

Schirrmeister

Springenberg

Fiederer

et al. 2017

Human Brain Mapping

2,272

2,128

View full text Add to dashboard Cite

Deep learning with convolutional neural networks (deep ConvNets) has revolutionized computer vision through end‐to‐end learning, that is, learning from the raw data. There is increasing interest in using deep ConvNets for end‐to‐end EEG analysis, but a better understanding of how to design and train ConvNets for end‐to‐end EEG decoding and how to visualize the informative EEG features the ConvNets learn is still needed. Here, we studied deep ConvNets with a range of different architectures, designed for decoding imagined or executed tasks from raw EEG. Our results show that recent advances from the machine learning field, including batch normalization and exponential linear units, together with a cropped training strategy, boosted the deep ConvNets decoding performance, reaching at least as good performance as the widely used filter bank common spatial patterns (FBCSP) algorithm (mean decoding accuracies 82.1% FBCSP, 84.0% deep ConvNets). While FBCSP is designed to use spectral power modulations, the features used by ConvNets are not fixed a priori. Our novel methods for visualizing the learned features demonstrated that ConvNets indeed learned to use spectral power modulations in the alpha, beta, and high gamma frequencies, and proved useful for spatially mapping the learned features by revealing the topography of the causal contributions of features in different frequency bands to the decoding decision. Our study thus shows how to design and train ConvNets to decode task‐related information from the raw EEG without handcrafted features and highlights the potential of deep ConvNets combined with advanced visualization techniques for EEG‐based brain mapping. Hum Brain Mapp 38:5391–5420, 2017. © 2017 Wiley Periodicals, Inc.

show abstract

Constructing long short-term memory based deep recurrent neural networks for large vocabulary speech recognition

Cited by 260 publications

References 47 publications

Automatic Speech Recognition with Deep Neural Networks for Impaired Speech

Automatic Speech Recognition with Deep Neural Networks for Impaired Speech

Deep learning with convolutional neural networks for decoding and visualization of EEG pathology

Deep learning with convolutional neural networks for EEG decoding and visualization

Contact Info

Product

Resources

About