COVID-MTL: Multitask learning with Shift3D and random-weighted loss for COVID-19 diagnosis and severity assessment

Abstract

There is an urgent need for automated methods to assist accurate and effective assessment of COVID-19. Radiology and nucleic acid test (NAT) are complementary COVID-19 diagnosis methods. In this paper, we present an end-to-end multitask learning (MTL) framework (COVID-MTL) that is capable of automated and simultaneous detection (against both radiology and NAT) and severity assessment of COVID-19. COVID-MTL learns different COVID-19 tasks in parallel through our novel random-weighted loss function, which assigns learning weights under Dirichlet distribution to prevent task dominance; our new 3D real-time augmentation algorithm (Shift3D) introduces space variances for 3D CNN components by shifting low-level feature representations of volumetric inputs in three dimensions; thereby, the MTL framework is able to accelerate convergence and improve joint learning performance compared to single-task models. By only using chest CT scans, COVID-MTL was trained on 930 CT scans and tested on separate 399 cases. COVID-MTL achieved AUCs of 0.939 and 0.846, and accuracies of 90.23% and 79.20% for detection of COVID-19 against radiology and NAT, respectively, which outperformed the state-of-the-art models. Meanwhile, COVID-MTL yielded AUC of 0.800 ± 0.020 and 0.813 ± 0.021 (with transfer learning) for classifying control/suspected, mild/regular, and severe/critically-ill cases. To decipher the recognition mechanism, we also identified high-throughput lung features that were significantly related (P < 0.001) to the positivity and severity of COVID-19.

An automated diagnosis and classification of COVID-19 from chest CT images using a transfer learning-based convolutional neural network

“…Guoqing et al [ 28 ] introduced a multitask learning (MTL) framework for COVID-19 automated diagnosis. Unsupervised lung segmentation, Shift3D, and a novel random-weighted loss function are used.…”

Test-time Adaptation with Calibration of Medical Image Classification Nets for Label Distribution Shift

“…The severity of COVID-19 is divided to 6 levels: control (S0), suspected (S1), mild (S2), regular (S3), severe (S4) and critically (S5). The preprocessing and automatic lung segmentation process are the same as a recent work [2] on this dataset. Experimental setting.…”

“…Evaluation metrics. For both tasks, the diagnosis performance is evaluated with accuracy, area under the receiver operating characteristic curve (AUC) and Obuchowski index (OI) [19], as reported in related works [2,5,21]. Considering the AUC is defined for binary classification while ours are multi-class classification tasks, we combine the classes and convert the multi-class classification to several binary classifications.…”

U-GAT: Multimodal Graph Attention Network for COVID-19 Outcome Prediction

“…In contrast to most other multitask methods, they branched the final feature maps of a U-Net architecture for the two classification tasks instead of the low-resolution bottleneck representation that most approaches used for classification, reporting an improved performance. Bao and Wang (2020) introduced a multitask framework built on a SqueezeNet (Iandola et al, 2016) backbone for the segmentation of the lung and two classification tasks: the detection of COVID-19 and the estimation of three degrees of severity. Similar to our approach Näppi et al (2021) use the bottleneck features of a pretrained U-Net for the prediction of COVID-19 progression and mortality.…”