Commercial AI solutions in detecting COVID-19 pneumonia in chest CT: not yet ready for clinical implementation?

Jungmann, Florian; Müller, Lukáš; Hähn, Felix; Weustenfeld, Maximilian; Dapper, Ann-Kathrin; Mähringer‐Kunz, Aline; Graafen, Dirk; Düber, Christoph; Schafigh, Darius; Santos, Daniel Pinto dos; Mildenberger, P.; Kloeckner, Roman

doi:10.1007/s00330-021-08409-4

Cited by 11 publications

(7 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Future implications of using computer tools might be implemented with fully automatic AI-based quantification software, such as Li L 2020 et al [ 33 ] and Jungmann F. et al [ 34 ] suggested in their study. This last one calculated the sensitivity, specificity, positive predictive value (PPV), the negative predictive value (NPV), and the area under the curve (AUC) for four AI solutions from different vendors compared to the visual score obtained by two radiologists to differentiate Sars-CoV-2 related pneumonia from other lung conditions.…”

Section: Discussionmentioning

confidence: 99%

Different Lung Parenchyma Quantification Using Dissimilar Segmentation Software: A Multi-Center Study for COVID-19 Patients

et al. 2022

View full text Add to dashboard Cite

Background: Chest Computed Tomography (CT) imaging has played a central role in the diagnosis of interstitial pneumonia in patients affected by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and can be used to obtain the extent of lung involvement in COVID-19 pneumonia patients either qualitatively, via visual inspection, or quantitatively, via AI-based software. This study aims to compare the qualitative/quantitative pathological lung extension data on COVID-19 patients. Secondly, the quantitative data obtained were compared to verify their concordance since they were derived from three different lung segmentation software. Methods: This double-center study includes a total of 120 COVID-19 patients (60 from each center) with positive reverse-transcription polymerase chain reaction (RT-PCR) who underwent a chest CT scan from November 2020 to February 2021. CT scans were analyzed retrospectively and independently in each center. Specifically, CT images were examined manually by two different and experienced radiologists for each center, providing the qualitative extent score of lung involvement, whereas the quantitative analysis was performed by one trained radiographer for each center using three different software: 3DSlicer, CT Lung Density Analysis, and CT Pulmo 3D. Results: The agreement between radiologists for visual estimation of pneumonia at CT can be defined as good (ICC 0.79, 95% CI 0.73–0.84). The statistical tests show that 3DSlicer overestimates the measures assessed; however, ICC index returns a value of 0.92 (CI 0.90–0.94), indicating excellent reliability within the three software employed. ICC was also performed between each single software and the median of the visual score provided by the radiologists. This statistical analysis underlines that the best agreement is between 3D Slicer “LungCTAnalyzer” and the median of the visual score (0.75 with a CI 0.67–82 and with a median value of 22% of disease extension for the software and 25% for the visual values). Conclusions: This study provides for the first time a direct comparison between the actual gold standard, which is represented by the qualitative information described by radiologists, and novel quantitative AI-based techniques, here represented by three different commonly used lung segmentation software, underlying the importance of these specific values that in the future could be implemented as consistent prognostic and clinical course parameters.

show abstract

Section: Discussionmentioning

confidence: 99%

Different Lung Parenchyma Quantification Using Dissimilar Segmentation Software: A Multi-Center Study for COVID-19 Patients

et al. 2022

View full text Add to dashboard Cite

show abstract

“…A clinical example of this model can be found in other studies, such as Suri J.S et al [21] and Gujot J. et al [22], in which the earlier cited software offered a valid tool to detect and classify affected patients. Nevertheless, Jungmann F. et al [23] stated their concern regarding the actual AI solutions, such as Icolung, as tools to assess positive predictive value (PPV). The aim of this study was to evaluate the performance of commercial AI solutions in differentiating COVID-19 pneumonia from other lung conditions.…”

Section: Discussionmentioning

confidence: 99%

Comparing Visual and Software-Based Quantitative Assessment Scores of Lungs’ Parenchymal Involvement Quantification in COVID-19 Patients

Nicolò,

Adraman,

Risoli

et al. 2024

Diagnostics

View full text Add to dashboard Cite

(1) Background: Computed tomography (CT) plays a paramount role in the characterization and follow-up of COVID-19. Several score systems have been implemented to properly assess the lung parenchyma involved in patients suffering from SARS-CoV-2 infection, such as the visual quantitative assessment score (VQAS) and software-based quantitative assessment score (SBQAS) to help in managing patients with SARS-CoV-2 infection. This study aims to investigate and compare the diagnostic accuracy of the VQAS and SBQAS with two different types of software based on artificial intelligence (AI) in patients affected by SARS-CoV-2. (2) Methods: This is a retrospective study; a total of 90 patients were enrolled with the following criteria: patients’ age more than 18 years old, positive test for COVID-19 and unenhanced chest CT scan obtained between March and June 2021. The VQAS was independently assessed, and the SBQAS was performed with two different artificial intelligence-driven software programs (Icolung and CT-COPD). The Intraclass Correlation Coefficient (ICC) statistical index and Bland–Altman Plot were employed. (3) Results: The agreement scores between radiologists (R1 and R2) for the VQAS of the lung parenchyma involved in the CT images were good (ICC = 0.871). The agreement score between the two software types for the SBQAS was moderate (ICC = 0.584). The accordance between Icolung and the median of the visual evaluations (Median R1–R2) was good (ICC = 0.885). The correspondence between CT-COPD and the median of the VQAS (Median R1–R2) was moderate (ICC = 0.622). (4) Conclusions: This study showed moderate and good agreement upon the VQAS and the SBQAS; enhancing this approach as a valuable tool to manage COVID-19 patients and the combination of AI tools with physician expertise can lead to the most accurate diagnosis and treatment plans for patients.

show abstract

“…[165] Specific examples include development of automated tools for interpreting CT scans, chest X-rays and lung ultrasound diagnostics, [166][167][168][169][170] though clinical trials to establish the benefits and safety of image analysis in clinical practice are awaited. [171]…”

Section: Digital Innovation To Manage Covid-19 Infectionsmentioning

confidence: 99%

Telemedicine and virtual respiratory care in the era of COVID-19

Pinnock

Murphie²,

Vogiatzis

et al. 2022

ERJ Open Res

View full text Add to dashboard Cite

The World Health Organization defines telemedicine as “an interaction between a health care provider and a patient when the two are separated by distance”. The COVID-19 pandemic has forced a dramatic shift to telephone and video consulting for follow up and routine ambulatory care for reasons of infection control. Short Message Service (“text”) messaging has proved a useful adjunct to remote consulting allowing transfer of photographs and documents. Maintaining non-communicable diseases care is a core component of pandemic preparedness and telemedicine has developed to enable (for example) remote monitoring of sleep apnoea, telemonitoring of chronic obstructive pulmonary disease, digital support for asthma self-management, remote delivery of pulmonary rehabilitation. There are multiple exemplars of telehealth instigated rapidly to provide care for people with COVID-19, to manage the spread of the pandemic, or to maintain safe routine diagnostic or treatment services.Despite many positive examples of equivalent functionality and safety, there remain questions about the impact of remote delivery of care on rapport and the longer-term impact on patient/professional relationships. Although telehealth has the potential to contribute to universal health coverage by providing cost-effective accessible care, there is a risk of increasing social health inequalities if the “digital divide” excludes those most in need of care. As we emerge from the pandemic, the balance of remote versus face-to-face consulting, and the specific role of digital health in different clinical and healthcare contexts will evolve. What is clear is that telemedicine in one form or another will be part of the “new norm”.

show abstract

Commercial AI solutions in detecting COVID-19 pneumonia in chest CT: not yet ready for clinical implementation?

Cited by 11 publications

References 24 publications

Different Lung Parenchyma Quantification Using Dissimilar Segmentation Software: A Multi-Center Study for COVID-19 Patients

Different Lung Parenchyma Quantification Using Dissimilar Segmentation Software: A Multi-Center Study for COVID-19 Patients

Comparing Visual and Software-Based Quantitative Assessment Scores of Lungs’ Parenchymal Involvement Quantification in COVID-19 Patients

Telemedicine and virtual respiratory care in the era of COVID-19

Contact Info

Product

Resources

About