End‐to‐end deep learning classification of vocal pathology using stacked vowels

Abstract

ObjectivesAdvances in artificial intelligence (AI) technology have increased the feasibility of classifying voice disorders using voice recordings as a screening tool. This work develops upon previous models that take in single vowel recordings by analyzing multiple vowel recordings simultaneously to enhance prediction of vocal pathology.MethodsVoice samples from the Saarbruecken Voice Database, including three sustained vowels (/a/, /i/, /u/) from 687 healthy human participants and 334 dysphonic patients, were used to train 1‐dimensional convolutional neural network models for multiclass classification of healthy, hyperfunctional dysphonia, and laryngitis voice recordings. Three models were trained: (1) a baseline model that analyzed individual vowels in isolation, (2) a stacked vowel model that analyzed three vowels (/a/, /i/, /u/) in the neutral pitch simultaneously, and (3) a stacked pitch model that analyzed the /a/ vowel in three pitches (low, neutral, and high) simultaneously.ResultsFor multiclass classification of healthy, hyperfunctional dysphonia, and laryngitis voice recordings, the stacked vowel model demonstrated higher performance compared with the baseline and stacked pitch models (F1 score 0.81 vs. 0.77 and 0.78, respectively). Specifically, the stacked vowel model achieved higher performance for class‐specific classification of hyperfunctional dysphonia voice samples compared with the baseline and stacked pitch models (F1 score 0.56 vs. 0.49 and 0.50, respectively).ConclusionsThis study demonstrates the feasibility and potential of analyzing multiple sustained vowel recordings simultaneously to improve AI‐driven screening and classification of vocal pathology. The stacked vowel model architecture in particular offers promise to enhance such an approach.Lay SummaryAI analysis of multiple vowel recordings can improve classification of voice pathologies compared with models using a single sustained vowel and offer a strategy to enhance AI‐driven screening of voice disorders.Level of Evidence3

A Comprehensive Study of Audio Profiling: Methods, Applications, Challenges, and Future Directions

“…Apart from these, there are different end-to-end approaches that handle all aspects of an audio profiling process, from the initial input to the final output, without requiring manual intervention or separate processing stages. Raw waveforms are taken as an input and different audio profiling tasks such as age estimation [36] [37], voice pathology detection [38], Speech Emotion Recognition [39], acoustic scene classification [40] [41] are obtained as output directly. End-to-end systems perform automatic feature extraction learning directly from raw inputs without manual intervention.…”