Hierarchical Multi-modal Transformer for Automatic Detection of COVID-19

Tang, Shuyun; Hu, Xinying; Atlas, Les; Khanzada, Amil; Pilancı, Mert

doi:10.1145/3556384.3556414

Cited by 5 publications

(3 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From the obtained results, the authors concluded that there is a considerable improvement in COVID-19 detection. Similar studies can be found in [26,29,44,59,63].…”

Section: Covid-19 Firstly Reported In 2019 In Wuhansupporting

confidence: 87%

Multi-modal approach for COVID-19 detection using coughs and self-reported symptoms

Nguyen-Trong¹,

Nguyen-Hoang²

2023

IFS

View full text Add to dashboard Cite

COVID-19 (Coronavirus Disease of 2019) is one of the most challenging healthcare crises of the twenty-first century. The pandemic causes many negative impacts on all aspects of life and livelihoods. Although recent developments of relevant vaccines, such as Pfizer/BioNTech mRNA, AstraZeneca, or Moderna, the emergence of new virus mutations and their fast infection rate yet pose significant threats to public health. In this context, early detection of the disease is an important factor to reduce its effect and quickly control the spread of pandemic. Nevertheless, many countries still rely on methods that are either expensive and time-consuming (i.e., Reverse-transcription polymerase chain reaction) or uncomfortable and difficult for self-testing (i.e., Rapid Antigen Test Nasal). Recently, deep learning methods have been proposed as a potential solution for COVID-19 analysis. However, previous works usually focus on a single symptom, which can omit critical information for disease diagnosis. Therefore, in this study, we propose a multi-modal method to detect COVID-19 using cough sounds and self-reported symptoms. The proposed method consists of five neural networks to deal with different input features, including CNN-biLSTM for MFCC features, EfficientNetV2 for Mel spectrogram images, MLP for self-reported symptoms, C-YAMNet for cough detection, and RNNoise for noise-canceling. Experimental results demonstrated that our method outperformed the other state-of-the-art methods with a high AUC, accuracy, and F1-score of 98.6%, 96.9%, and 96.9% on the testing set.

show abstract

“…From the obtained results, the authors concluded that there is a considerable improvement in COVID-19 detection. Similar studies can be found in [26,29,44,59,63].…”

Section: Covid-19 Firstly Reported In 2019 In Wuhansupporting

confidence: 87%

Multi-modal approach for COVID-19 detection using coughs and self-reported symptoms

Nguyen-Trong¹,

Nguyen-Hoang²

2023

IFS

View full text Add to dashboard Cite

show abstract

“…The Transformer [116] has become popular due to its proven efficacy in numerous applications, including pandemic response [43,[117][118][119]. We can define the edge weights in the bipartite graph using learned embeddings from the Graph Transformer [120][121][122][123][124][125].…”

Section: Rank-based Approachmentioning

confidence: 99%

Privacy-Enhancing Digital Contact Tracing with Machine Learning for Pandemic Response: A Comprehensive Review

Hang

Tsai

et al. 2023

BDCC

View full text Add to dashboard Cite

The rapid global spread of the coronavirus disease (COVID-19) has severely impacted daily life worldwide. As potential solutions, various digital contact tracing (DCT) strategies have emerged to mitigate the virus’s spread while maintaining economic and social activities. The computational epidemiology problems of DCT often involve parameter optimization through learning processes, making it crucial to understand how to apply machine learning techniques for effective DCT optimization. While numerous research studies on DCT have emerged recently, most existing reviews primarily focus on DCT application design and implementation. This paper offers a comprehensive overview of privacy-preserving machine learning-based DCT in preparation for future pandemics. We propose a new taxonomy to classify existing DCT strategies into forward, backward, and proactive contact tracing. We then categorize several DCT apps developed during the COVID-19 pandemic based on their tracing strategies. Furthermore, we derive three research questions related to computational epidemiology for DCT and provide a detailed description of machine learning techniques to address these problems. We discuss the challenges of learning-based DCT and suggest potential solutions. Additionally, we include a case study demonstrating the review’s insights into the pandemic response. Finally, we summarize the study’s limitations and highlight promising future research directions in DCT.

show abstract

“…In [35,[46][47][48][49], the ML architecture and hyperparameters were optimized to obtain deep features with convolutional neural networks (CNNsƒ) layers to address the problems of COVID-19 or respiratory distress detection. These deep features can be concatenated and input to a neural network-based classifier trained on an end-to-end basis to combine the parameters.…”

Section: Voice-based Estimation Of Respiratory Distressmentioning

confidence: 99%

Automatic Detection of Dyspnea in Real Human–Robot Interaction Scenarios

Alvarado,

Grágeda,

Luzanto

et al. 2023

Sensors

View full text Add to dashboard Cite

A respiratory distress estimation technique for telephony previously proposed by the authors is adapted and evaluated in real static and dynamic HRI scenarios. The system is evaluated with a telephone dataset re-recorded using the robotic platform designed and implemented for this study. In addition, the original telephone training data are modified using an environmental model that incorporates natural robot-generated and external noise sources and reverberant effects using room impulse responses (RIRs). The results indicate that the average accuracy and AUC are just 0.4% less than those obtained with matched training/testing conditions with simulated data. Quite surprisingly, there is not much difference in accuracy and AUC between static and dynamic HRI conditions. Moreover, the beamforming methods delay-and-sum and MVDR lead to average improvement in accuracy and AUC equal to 8% and 2%, respectively, when applied to training and testing data. Regarding the complementarity of time-dependent and time-independent features, the combination of both types of classifiers provides the best joint accuracy and AUC score.

show abstract

Hierarchical Multi-modal Transformer for Automatic Detection of COVID-19

Cited by 5 publications

References 8 publications

Multi-modal approach for COVID-19 detection using coughs and self-reported symptoms

Multi-modal approach for COVID-19 detection using coughs and self-reported symptoms

Privacy-Enhancing Digital Contact Tracing with Machine Learning for Pandemic Response: A Comprehensive Review

Automatic Detection of Dyspnea in Real Human–Robot Interaction Scenarios

Contact Info

Product

Resources

About