Dysarthric-speech detection using transfer learning with convolutional neural networks

Sekhar, S. R. Mani; Kashyap, Gaurav; Bhansali, Akshay; Abishek, A.; Singh, Kushan

doi:10.1016/j.icte.2021.07.004

Cited by 19 publications

(13 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A crucial reason is that the data is difficult to meet the needs of state-of-the-art end-to-end recognition methods. Therefore, some studies have tried solutions such as data augmentation [ 139 , 140 , 149 , 171 ] and transfer learning [ 141 , 145 , 149 ] to solve this problem. The details of the diagnosis systems in these studies, including data sources, voice type, voice feature, classifier, and effect, can be found in Table 4 .…”

Section: Pathological Voice Recognition For Diagnosis and Evaluationmentioning

confidence: 99%

“…DL accounts for only a small part of the current pathological voice recognition methods because a large amount of data is unavailable [ 92 , 93 , 101 , 104 , 110 , [119] , [124] , 129 , 145 , 146 ]. However, researchers can continuously explore the potential of DL methods such as Attention-based LSTM [ 45 , 47 ], end-to-end models [ 134 ], and Transformer models [ 187 ] and try advanced recognition algorithms to improve the performance of IST for medical applications.…”

Section: Limitations and Future Directionsmentioning

confidence: 99%

See 1 more Smart Citation

Intelligent speech technologies for transcription, disease diagnosis, and medical equipment interactive control in smart hospitals: A review

Zhang

Qiu

et al. 2023

Computers in Biology and Medicine

View full text Add to dashboard Cite

Section: Pathological Voice Recognition For Diagnosis and Evaluationmentioning

confidence: 99%

Section: Limitations and Future Directionsmentioning

confidence: 99%

Intelligent speech technologies for transcription, disease diagnosis, and medical equipment interactive control in smart hospitals: A review

Zhang

Qiu

et al. 2023

Computers in Biology and Medicine

View full text Add to dashboard Cite

“…Other studies were focused on the detection of Dysarthria [11,12]. Dysarthria is a motor impairment preventing a person from articulating speech correctly.…”

Section: Introductionmentioning

confidence: 99%

“…These studies generally use TORGO dataset [13] to train and validate their models which consists of audios of varying lengths and of different categories: non-words, short words, restricted sentences, and unrestricted sentences. Several DL models were used, such as Mel-Spectrograms as input to a 2D CNN network [11] or a GAN model with spectral features [12].…”

Section: Introductionmentioning

confidence: 99%

Classification of Speech Arrests and Speech Impairments during Awake Craniotomy: a multi-databases analysis

Maoudj,

Kuwano,

Panheleux

et al. 2024

Preprint

View full text Add to dashboard Cite

Purpose: Awake craniotomy presents a unique opportunity to map and preserve critical brain functions, particularly speech, during tumor resection. The ability to accurately assess linguistic functions in real-time not only enhances surgical precision but also contributes significantly to improving postoperative outcomes. However, today, its evaluation is subjective as it relies on a clinician’s observations only. This paper explores the use of a deep learning based model for the objective assessment of speech arrest and speech impairments during awake craniotomy. Methods: We extracted 1883 3-second audio clips containing the patient’s response following Direct Electrical Stimulation from 23 awake craniotomies recorded from two operating rooms of the Tokyo Women’s Medical University Hospital (Japan) and 2 awake craniotomies recorded from the University Hospital of Brest (France). A Wav2Vec2-based model has been trained and used to detect speech arrests and speech impairments. Experiments were performed with different datasets settings and preprocessing techniques and the performances of the model were evaluated using the F1-score. Results: The F1-score was 84.12% when the model was trained and tested on Japanese data only. In a cross-language situation, the F1-score was 74.68% when the model was trained on Japanese data and tested on French data. Conclusion: The results are encouraging even in a cross-language situation but further evaluation is required. The integration of preprocessing techniques, in particular noise reduction, improved the results significantly.

show abstract

“…In 2021, S R Mani et al [15] proposed a transfer learningbased convolutional neural network model (TL-CNN) and converted audio samples to Mel-spectrograms to improve accuracy on the TORGO dataset. When compared to other machine learning models, the proposed TL-CNN achieved improved accuracy.…”

mentioning

confidence: 99%

Re-Talk: Automated Speech Assistance for People with Dysarthria

Ali

Hassan²,

Salah³

et al. 2023

النشرة المعلوماتیة فی الحاسبات والمعلومات

View full text Add to dashboard Cite

Dysarthria is a speech motor disorder where the muscles responsible for speech production, such as in the face, mouth, or respiratory system, have trouble coordinating and controlling themselves. Our research goal is to help individuals with dysarthria communicate effectively. Often, physical conditions make it challenging for them to express their thoughts through writing. Our research introduces an automatic speech assistant solution, consisting of two main parts: speech recognition and auto-correct. The speech recognition component takes the person's distorted speech as input, converts it to text, and then sends it to the auto-correct module to fix any mistakes or unclear words. We tested our model on both English and Arabic datasets. The English dataset showed a 50% Word Error Rate (WER) which was reduced to 40% after using the auto-correct module. Our results outperformed previous studies by 4.5%. However, the WER on the Arabic dataset was 80% which is not a satisfactory result, due to the limited size of the Egyptian Dialect Dysarthric Speech (EDDS) database.

show abstract

Dysarthric-speech detection using transfer learning with convolutional neural networks

Cited by 19 publications

References 5 publications

Intelligent speech technologies for transcription, disease diagnosis, and medical equipment interactive control in smart hospitals: A review

Intelligent speech technologies for transcription, disease diagnosis, and medical equipment interactive control in smart hospitals: A review

Classification of Speech Arrests and Speech Impairments during Awake Craniotomy: a multi-databases analysis

Re-Talk: Automated Speech Assistance for People with Dysarthria

Contact Info

Product

Resources

About