COVID-19 score severity prediction using 3D-based deep learning models on lung ultrasound video: could the system stand the test of time and of disease’s evolution?

Abstract

The coronavirus disease 2019 (COVID-19) emerged in December 2019 and rapidly evolved into a global pandemic affecting more than 750 million people and causing more than 6,8 million deaths worldwide. The most important and often deadly manifestation of the disease is pneumonia. Among other imaging modalities, lung ultrasound (LUS) represents an efficient tool for management of COVID-19 patients, as it can be used to classify and monitor lung involvement, as well as to assess disease's severity. However, LUS requires a skilled sonographer to interpret findings. For this reason, we developed a Deep Learning (DL)-powered system, for multi-class automatic assessment of LUS videos in COVID-19 patients. We trained and tested (70,4% accuracy) our system with LUS images acquired from patients of the so-called first and second waves (2020). However, lung disease induced by COVID-19 is changed with time due to virus behavior, presence of vaccines and availability of therapies. Accordingly, we tested again performances of our system with LUS images of COVID-19 patients enrolled in 2022. Surprisingly, the performances of the system dramatically drop to an unsatisfiable accuracy of 48,3%. We speculate that this fact is primarily due to the differences in clinical features of patients hospitalized in 2020 and 2022. However, these findings also highlight a critical point on the management of DL systems designed for prognosis and diagnosis of rapidly-changing diseases. Accordingly, a continuous training update should be considered as the only viable strategy to preserve DL systems reliability and to improve the credibility of DL systems from the medical community.