Automatic Diagnosis of COVID-19 Patients from Unstructured Data Based on a Novel Weighting Scheme

Abstract

The extraction of features from unstructured clinical data of Covid-19 patients is critical for guiding clinical decision-making and diagnosing this viral disease. Furthermore, an early and accurate diagnosis of COVID-19 can reduce the burden on healthcare systems. In this paper, an improved Term Weighting technique combined with Parts-Of-Speech (POS) Tagging is proposed to reduce dimensions for automatic and effective classification of clinical text related to Covid-19 disease. Term Frequency-Inverse Document Frequency (TF-IDF) is the most often used term weighting scheme (TWS). However, TF-IDF has several developments to improve its drawbacks, in particular, it is not efficient enough to classify text by assigning effective weights to the terms in unstructured data. In this research, we proposed a modification term weighting scheme: RTF-C-IEF and compare the proposed model with four extraction methods: TF, TF-IDF, TF-IHF, and TF-IEF. The experiment was conducted on two new datasets for COVID-19 patients. The first dataset was collected from government hospitals in Iraq with 3053 clinical records, and the second dataset with 1446 clinical reports, was collected from several different websites. Based on the experimental results using several popular classifiers applied to the datasets of Covid-19, we observe that the proposed scheme RTF-C-IEF achieves is a consistent performer with the best scores in most of the experiments. Further, the modified RTF-C-IEF proposed in the study outperformed the original scheme and other employed term weighting methods in most experiments. Thus, the proper selection of term weighting scheme among the different methods improves the performance of the classifier and helps to find the informative term.

An Intelligent Feature Selection Approach Based on a Novel Improve Binary Sparrow Search Algorithm for COVID-19 Classification

“…The RTF-C-IEF statistical weighting approach is used as the first step in the feature selection process for text mining to assess the importance of a term in a document. Since bag of words (BoW) and TF-IDF are less accurate than RTF-C-IEF, it was chosen as the feature extraction method [37]. RTF-C-IEF converts texts into vectors so that machine learning can process the COVID-19 clinical content properly.…”

An Integrated Framework for Early Differential Diagnosis of COVID-19 using Improved Fuzzy Cognitive Map Approaches

“…Other works have focused on optimizing model hyperparameters and feature selection using novel metaheuristic algorithms like Binary Sparrow Search for COVID-19 patient data classification, resulting in improved performance over unoptimized versions (9) . Hybrid models combining techniques like Sparger Wolf Hawk Optimization with deep neural networks have also shown promise for COVID-19 assessment (10) . For text data, weighting schemes have been employed to extract informative features from clinical notes for COVID-19 identification and mortality prediction (11) .…”