A deep-learning based multimodal system for Covid-19 diagnosis using breathing sounds and chest X-ray images

Abstract

Covid-19 has become a deadly pandemic claiming more than three million lives worldwide. SARS-CoV-2 causes distinct pathomorphological alterations in the respiratory system, thereby acting as a biomarker to aid its diagnosis. A multimodal framework (Ai-CovScan) for Covid-19 detection using breathing sounds, chest X-ray (CXR) images, and rapid antigen test (RAnT) is proposed. Transfer Learning approach using existing deep-learning Convolutional Neural Network (CNN) based on Inception-v3 is combined with Multi-Layered Perceptron (MLP) to develop the CovScanNet model for reducing false-negatives. This model reports a preliminary accuracy of 80% for the breathing sound analysis, and 99.66% Covid-19 detection accuracy for the curated CXR image dataset. Based on Ai-CovScan, a smartphone app is conceptualised as a mass-deployable screening tool, which could alter the course of this pandemic. This app ’s deployment could minimise the number of people accessing the limited and expensive confirmatory tests, thereby reducing the burden on the severely stressed healthcare infrastructure.

COVID-19 prognosis using limited chest X-ray images

“…The individual nodes update the model according to the dataset that they have and their updated weights are averaged and the common server model is updated. In [42] a multimodal system is developed based on data consisting of breathing sounds and chest X-ray images. Sound data is converted to spectrograms and convolutional neural networks are used for analysis for both sound data and chest xray images.…”

Cov-Net: A computer-aided diagnosis method for recognizing COVID-19 from chest X-ray images via machine vision

“…Specifically, LeNet-5 framework is employed as the backbone to learn features from X-ray images, and a single-layer extreme learning machine serves as the classifier to differentiate between normal case, viral pneumonia, and COVID-19. A deep-learning based multi-modal system is proposed in Sait et al (2021) , which makes use of not only CXR images but also the breathing sounds for COVID-19 diagnosis, whereas the feature extractor therein is the CNN model as well.

Fig.

…”

Detection of COVID-19 in smartphone-based breathing recordings: A pre-screening deep learning tool

“…The majority of studies have used coughing sounds to train deep learning networks. In addition, only two studies have utilized breathing sounds as input to the trained models [ 68 , 69 ]. The only limitation in [ 69 ] is the heavy unbalance in favor of the normal subjects against COVID-19 subjects, which could have been the reason behind the high performance metrics achieved.…”

“…The only limitation in [ 69 ] is the heavy unbalance in favor of the normal subjects against COVID-19 subjects, which could have been the reason behind the high performance metrics achieved. In addition, authors in [ 68 ] use only 5 COVID-19 subjects, which does not ensure a generalized performance of deep learning networks. In contrary, the proposed study utilized a more balanced dataset with 120 COVID-19 subjects and the performance was higher than most of other studies.…”