Automatic Recognition, Segmentation, and Sex Assignment of Nocturnal Asthmatic Coughs and Cough Epochs in Smartphone Audio Recordings: Observational Field Study

Abstract

Background Asthma is one of the most prevalent chronic respiratory diseases. Despite increased investment in treatment, little progress has been made in the early recognition and treatment of asthma exacerbations over the last decade. Nocturnal cough monitoring may provide an opportunity to identify patients at risk for imminent exacerbations. Recently developed approaches enable smartphone-based cough monitoring. These approaches, however, have not undergone longitudinal overnight testing nor have they been specifically evaluated in the context of asthma. Also, the problem of distinguishing partner coughs from patient coughs when two or more people are sleeping in the same room using contact-free audio recordings remains unsolved. Objective The objective of this study was to evaluate the automatic recognition and segmentation of nocturnal asthmatic coughs and cough epochs in smartphone-based audio recordings that were collected in the field. We also aimed to distinguish partner coughs from patient coughs in contact-free audio recordings by classifying coughs based on sex. Methods We used a convolutional neural network model that we had developed in previous work for automated cough recognition. We further used techniques (such as ensemble learning, minibatch balancing, and thresholding) to address the imbalance in the data set. We evaluated the classifier in a classification task and a segmentation task. The cough-recognition classifier served as the basis for the cough-segmentation classifier from continuous audio recordings. We compared automated cough and cough-epoch counts to human-annotated cough and cough-epoch counts. We employed Gaussian mixture models to build a classifier for cough and cough-epoch signals based on sex. Results We recorded audio data from 94 adults with asthma (overall: mean 43 years; SD 16 years; female: 54/94, 57%; male 40/94, 43%). Audio data were recorded by each participant in their everyday environment using a smartphone placed next to their bed; recordings were made over a period of 28 nights. Out of 704,697 sounds, we identified 30,304 sounds as coughs. A total of 26,166 coughs occurred without a 2-second pause between coughs, yielding 8238 cough epochs. The ensemble classifier performed well with a Matthews correlation coefficient of 92% in a pure classification task and achieved comparable cough counts to that of human annotators in the segmentation of coughing. The count difference between automated and human-annotated coughs was a mean –0.1 (95% CI –12.11, 11.91) coughs. The count difference between automated and human-annotated cough epochs was a mean 0.24 (95% CI –3.67, 4.15) cough epochs. The Gaussian mixture model cough epoch–based sex classification performed best yielding an accuracy of 83%. Conclusions Our study showed longitudinal nocturnal cough and cough-epoch recognition from nightly recorded smartphone-based audio from adults with asthma. The model distinguishes partner cough from patient cough in contact-free recordings by identifying cough and cough-epoch signals that correspond to the sex of the patient. This research represents a step towards enabling passive and scalable cough monitoring for adults with asthma.

Novel COVID-19 Screening Using Cough Recordings of A Mobile Patient Monitoring System

“…It is noted that many published results have shown very high specificity (90.9% -99.8%) at the cost of significantly lower sensitivity (70.5% -88%) [7], [19]. Although the models introduced in [8] has shown an overall accuracy of 99.8%, its dataset only consists of night-time recordings of participants during their sleep, which include very little noise to cope with. Some other studies have achieved similar model performance only with the addition of other device data, such as contact microphones, respiratory inductance plethysmography, ECG sensors, and accelerometers [20].…”

“…Besides the noise handling, audio segmentation and accurate extraction of cough episodes is a pivotal algorithmic step that can characterize the cough signatures and effectively quantify associated metrics such as cough frequency, cough duration, cough intensity, etc. There have been some research reports on cough segmentation and extraction [7], [8]. However, their algorithmic performances have been often reported based on simple counting of cough episodes and evaluation metrics for classification tasks rather than conducting systematic performance evaluations involving episodic and duration performance metrics.…”

<p>Nocturnal Cough and Sleep Quality to Assess Asthma Control and Predict Attacks</p>

“…For example, we were able to show that deep learning models trained on data from this study can detect and count nocturnal cough automatically with a comparable accuracy to human annotators. 36 …”

“…For example, we were able to show that deep learning models trained on data from this study can detect and count nocturnal cough automatically with a comparable accuracy to human annotators. 36 However, the accuracies presented in this paper for detecting and predicting relevant asthma outcomes do not warrant the use of nocturnal cough and sleep quality as selfcontained biomarkers for asthma management. The limiting factors for marker performance remain unclear.…”

Deep Neural Network-Based Respiratory Pathology Classification Using Cough Sounds

“…There is another line of research inquiry which mainly focuses on cough event detection (i.e., to identify the presence of cough events) in audio recordings [ 19 , 20 , 21 , 22 , 23 , 24 ]; however, in this investigation, we manually segment the cough epochs, and thus review of such studies is outside the scope of this report. Having said that, with the advent of deep learning, there is good progress made in the cough event detection from smartphone recordings, and incorporating such techniques at the preprocessing stage in the cough screening system could bypass the tedious manual segmentation process altogether [ 25 , 26 , 27 ].…”