The voice of COVID-19: Acoustic correlates of infection in sustained vowels

Bartl-Pokorny, Katrin D.; Pokorny, Florian B.; Batliner, Anton; Amiriparian, Shahin; Semertzidou, Anastasia; Eyben, Florian; Kramer, Elena; Schmidt, Florian H.; Schönweiler, R.; Wehler, Markus; Schuller, Björn W.

doi:10.1121/10.0005194

Cited by 39 publications

(21 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When analysing the acoustic peculiarities of patients with diseases that affect the anatomical correlates of speech production, the related studies reported that the peculiar acoustic parameters of the patients’ speech include fundamental frequency ( f o ), vowel formants, jitter, shimmer, HNR, and maximum phonation time (MPT) (Balamurali et al, 2020; Dogan et al, 2007; Jesus et al, 2015; Petrović-Lazić et al, 2011). Additionally, the peculiar acoustic parameters of voice samples of COVID-19 positive and COVID-19 negative participants were reported to include f o standard deviation, jitter, shimmer, HNR, the difference between the first two harmonic amplitudes (H1–H2), MPT, cepstral peak prominence (Asiaee et al, 2020), mean voiced segment length, and the number of voiced segments per second (Bartl-Pokorny et al, 2021). We can find that there are common acoustic peculiarities between the voice of COVID-19 patients and patients with some other diseases: f o -related features, jitter, shimmer, HNR, and MPT.…”

Section: Discussionmentioning

confidence: 99%

“…Several studies have reported acoustic peculiarities in the speech of patients who have diseases associated with symptoms affecting anatomical correlates of speech production, such as bronchial asthma (Balamurali et al, 2020;Dogan et al, 2007) or vocal cord disorders (Falk et al, 2021;Jesus et al, 2015;Petrović-Lazić et al, 2011). Differences in various acoustic parameters were also found in recent studies comparing speech samples of COVID-19 positive and COVID-19 negative individuals (Asiaee et al, 2020;Bartl-Pokorny et al, 2021). Motivated by acoustic voice peculiarities found for various diseases, machine learning has been increasingly applied to automatically detect medical conditions from voice, such as upper respiratory tract infection (Albes et al, 2020), Parkinson's disease (Yaman et al, 2020), and depression (Ringeval et al, 2019).…”

Section: Disease Detection Based On Bioacoustic Signalsmentioning

confidence: 91%

See 1 more Smart Citation

The Acoustic Dissection of Cough: Diving into Machine Listening-based COVID-19 Analysis and Detection

Ren

Chang

Bartl-Pokorny

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Purpose: The coronavirus disease 2019 (COVID-19) has caused a crisis worldwide. Amounts of efforts have been made to prevent and control COVID-19's transmission, from early screenings to vaccinations and treatments. Recently, due to the spring up of many automatic disease recognition applications based on machine listening techniques, it would be fast and cheap to detect COVID-19 from recordings of cough, a key symptom of COVID-19. To date, knowledge on the acoustic characteristics of COVID-19 cough sounds is limited, but would be essential for structuring effective and robust machine learning models. The present study aims to explore acoustic features for distinguishing COVID-19 positive individuals from COVID-19 negative ones based on their cough sounds. Methods: With the theory of computational paralinguistics, we analyse the acoustic correlates of COVID-19 cough sounds based on the COMPARE feature set, i. e., a standardised set of 6,373 acoustic higher-level features. Furthermore, we train automatic COVID-19 detection models with machine learning methods and explore the latent features by evaluating the contribution of all features to the COVID-19 status predictions. Results: The experimental results demonstrate that a set of acoustic parameters of cough sounds, e. g., statistical functionals of the root mean square energy and Mel-frequency cepstral coefficients, are relevant for the differentiation between COVID-19 positive and COVID-19 negative cough samples. Our automatic COVID-19 detection model performs significantly above chance level, i. e., at an unweighted average recall (UAR) of 0.632, on a data set consisting of 1,411 cough samples (COVID-19 positive/negative: 210/1,201). Conclusions: Based on the acoustic correlates analysis on the COMPARE feature set and the feature analysis in the effective COVID-19 detection model, we find that the machine learning method to a certain extent relies on acoustic features showing higher effects in conventional group difference testing.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Disease Detection Based On Bioacoustic Signalsmentioning

confidence: 91%

The Acoustic Dissection of Cough: Diving into Machine Listening-based COVID-19 Analysis and Detection

Ren

Chang

Bartl-Pokorny

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The sample spectrograms of the negative class of both datasets show a harmonic broad-band pattern with a quite similar width. Contrarily, the positive class is characterized by disruptions in the amplitude for distinct frequency ranges, indicating discontinuities in the pulmonic airstream during phonation in COVID-19 positive participants (Bartl-Pokorny et al, 2021 ). The impact of each segment of the Mel spectrograms on the output of the DNN model is visualized with help of SHAP values.…”

Section: Methodsmentioning

confidence: 99%

DeepSpectrumLite: A Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data

Amiriparian

Hübner

Karas

et al. 2022

Front. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

Deep neural speech and audio processing systems have a large number of trainable parameters, a relatively complex architecture, and require a vast amount of training data and computational power. These constraints make it more challenging to integrate such systems into embedded devices and utilize them for real-time, real-world applications. We tackle these limitations by introducing DeepSpectrumLite, an open-source, lightweight transfer learning framework for on-device speech and audio recognition using pre-trained image Convolutional Neural Networks (CNNs). The framework creates and augments Mel spectrogram plots on the fly from raw audio signals which are then used to finetune specific pre-trained CNNs for the target classification task. Subsequently, the whole pipeline can be run in real-time with a mean inference lag of 242.0 ms when a DenseNet121 model is used on a consumer-grade Motorola moto e7 plus smartphone. DeepSpectrumLite operates decentralized, eliminating the need for data upload for further processing. We demonstrate the suitability of the proposed transfer learning approach for embedded audio signal processing by obtaining state-of-the-art results on a set of paralinguistic and general audio tasks, including speech and music emotion recognition, social signal processing, COVID-19 cough and COVID-19 speech analysis, and snore sound classification. We provide an extensive command-line interface for users and developers which is comprehensively documented and publicly available at https://github.com/DeepSpectrum/DeepSpectrumLite.

show abstract

“…Recently some other methods were developed based on other physical phenomena, for example, based on quality of voice when a sustained vowel/a/ was pronounced and analyzed [4] , and a series of vowels/i:/,/e:/,/o:/,/u:/, and/a:/ was recorded and analyzed [5] . Similar to voice, speech is also an approach to diagnosing COVID-19.…”

Section: Introductionmentioning

confidence: 99%

Diagnosis of COVID-19 via acoustic analysis and artificial intelligence by monitoring breath sounds on smartphones

Chen

Wang

et al. 2022

Journal of Biomedical Informatics

View full text Add to dashboard Cite

The voice of COVID-19: Acoustic correlates of infection in sustained vowels

Cited by 39 publications

References 19 publications

The Acoustic Dissection of Cough: Diving into Machine Listening-based COVID-19 Analysis and Detection

The Acoustic Dissection of Cough: Diving into Machine Listening-based COVID-19 Analysis and Detection

DeepSpectrumLite: A Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data

Diagnosis of COVID-19 via acoustic analysis and artificial intelligence by monitoring breath sounds on smartphones

Contact Info

Product

Resources

About