How should studies using AI be reported? lessons from a systematic review in cardiac MRI

Maiter, Ahmed; Salehi, Mahan; Swift, Andrew J.; Alabed, Samer

doi:10.3389/fradi.2023.1112841

Cited by 5 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, different DL applications have been developed to automate the segmentation of cardiac structures on imaging and may help to improve efficiency and reliability ( 1 ). In a previous systematic review in 2022, 209 studies were included for AI-based cardiac MRI segmentation ( 17 , 18 ). However, our systematic review of DL-based cardiac CT segmentations identified only 18 studies.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Advancements in cardiac structures segmentation: a comprehensive systematic review of deep learning in CT imaging

Alnasser,

Abdulaal,

Maiter

et al. 2024

Front. Cardiovasc. Med.

Self Cite

View full text Add to dashboard Cite

BackgroundSegmentation of cardiac structures is an important step in evaluation of the heart on imaging. There has been growing interest in how artificial intelligence (AI) methods—particularly deep learning (DL)—can be used to automate this process. Existing AI approaches to cardiac segmentation have mostly focused on cardiac MRI. This systematic review aimed to appraise the performance and quality of supervised DL tools for the segmentation of cardiac structures on CT.MethodsEmbase and Medline databases were searched to identify related studies from January 1, 2013 to December 4, 2023. Original research studies published in peer-reviewed journals after January 1, 2013 were eligible for inclusion if they presented supervised DL-based tools for the segmentation of cardiac structures and non-coronary great vessels on CT. The data extracted from eligible studies included information about cardiac structure(s) being segmented, study location, DL architectures and reported performance metrics such as the Dice similarity coefficient (DSC). The quality of the included studies was assessed using the Checklist for Artificial Intelligence in Medical Imaging (CLAIM).Results18 studies published after 2020 were included. The DSC scores median achieved for the most commonly segmented structures were left atrium (0.88, IQR 0.83–0.91), left ventricle (0.91, IQR 0.89–0.94), left ventricle myocardium (0.83, IQR 0.82–0.92), right atrium (0.88, IQR 0.83–0.90), right ventricle (0.91, IQR 0.85–0.92), and pulmonary artery (0.92, IQR 0.87–0.93). Compliance of studies with CLAIM was variable. In particular, only 58% of studies showed compliance with dataset description criteria and most of the studies did not test or validate their models on external data (81%).ConclusionSupervised DL has been applied to the segmentation of various cardiac structures on CT. Most showed similar performance as measured by DSC values. Existing studies have been limited by the size and nature of the training datasets, inconsistent descriptions of ground truth annotations and lack of testing in external data or clinical settings.Systematic Review Registration[www.crd.york.ac.uk/prospero/], PROSPERO [CRD42023431113].

show abstract

Section: Discussionmentioning

confidence: 99%

“…Each included study was assessed for compliance with the criteria of the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) ( 16 ). The 42 individual criteria were divided into four domains: study description, dataset description, model description and model performance ( 17 , 18 ).…”

Section: Methodsmentioning

confidence: 99%

Advancements in cardiac structures segmentation: a comprehensive systematic review of deep learning in CT imaging

Alnasser,

Abdulaal,

Maiter

et al. 2024

Front. Cardiovasc. Med.

Self Cite

View full text Add to dashboard Cite

show abstract

“…These two studies used publicly available Kaggle datasets for the detection of acute and chronic PE. Although the proportions of acute and chronic PE cases are available from the original data source, these were not stated by the studies themselves, limiting their transparency—it is best practice for publications to provide all relevant clinical characteristics regardless of whether they can be accessed elsewhere ( 17 , 18 ). Public datasets utilised in the studies may not have had all the elements or features required for an accurate PE diagnosis, which may limit the model's ability to identify the spectrum of abnormalities related to PE.…”

Section: Discussionmentioning

confidence: 99%

“…Extracted data included study information (such as location, year and journal type), study design, data selection (such as number of participants, number of CTEPH cases and inclusion criteria), and the AI model being presented (such as validation and performance results). The quality of each included study was appraised by checking compliance with the individual criteria of the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) ( 16 ), which were divided into four domains ( 17 , 18 ).…”

Section: Methodsmentioning

confidence: 99%

A systematic review of artificial intelligence tools for chronic pulmonary embolism on CT pulmonary angiography

Abdulaal,

Maiter,

Salehi

et al. 2024

Front. Radiol.

Self Cite

View full text Add to dashboard Cite

BackgroundChronic pulmonary embolism (PE) may result in pulmonary hypertension (CTEPH). Automated CT pulmonary angiography (CTPA) interpretation using artificial intelligence (AI) tools has the potential for improving diagnostic accuracy, reducing delays to diagnosis and yielding novel information of clinical value in CTEPH. This systematic review aimed to identify and appraise existing studies presenting AI tools for CTPA in the context of chronic PE and CTEPH.MethodsMEDLINE and EMBASE databases were searched on 11 September 2023. Journal publications presenting AI tools for CTPA in patients with chronic PE or CTEPH were eligible for inclusion. Information about model design, training and testing was extracted. Study quality was assessed using compliance with the Checklist for Artificial Intelligence in Medical Imaging (CLAIM).ResultsFive studies were eligible for inclusion, all of which presented deep learning AI models to evaluate PE. First study evaluated the lung parenchymal changes in chronic PE and two studies used an AI model to classify PE, with none directly assessing the pulmonary arteries. In addition, a separate study developed a CNN tool to distinguish chronic PE using 2D maximum intensity projection reconstructions. While another study assessed a novel automated approach to quantify hypoperfusion to help in the severity assessment of CTEPH. While descriptions of model design and training were reliable, descriptions of the datasets used in training and testing were more inconsistent.ConclusionIn contrast to AI tools for evaluation of acute PE, there has been limited investigation of AI-based approaches to characterising chronic PE and CTEPH on CTPA. Existing studies are limited by inconsistent reporting of the data used to train and test their models. This systematic review highlights an area of potential expansion for the field of AI in medical image interpretation.There is limited knowledge of A systematic review of artificial intelligence tools for chronic pulmonary embolism in CT. This systematic review provides an assessment on research that examined deep learning algorithms in detecting CTEPH on CTPA images, the number of studies assessing the utility of deep learning on CTPA in CTEPH was unclear and should be highlighted.

show abstract

“…The development and application of deep learning methods is an active research topic in radiology 32 – 34 . Standards for the reporting of artificial intelligence methods were established for the medical field as a whole in 35 and the topic was specifically discussed for CMR in 36 , 37 .…”

Section: Discussionmentioning

confidence: 99%

Assessment of deep learning segmentation for real-time free-breathing cardiac magnetic resonance imaging at rest and under exercise stress

Schilling,

Unterberg-Buchwald,

Lotz

et al. 2024

Sci Rep

View full text Add to dashboard Cite

In recent years, a variety of deep learning networks for cardiac MRI (CMR) segmentation have been developed and analyzed. However, nearly all of them are focused on cine CMR under breathold. In this work, accuracy of deep learning methods is assessed for volumetric analysis (via segmentation) of the left ventricle in real-time free-breathing CMR at rest and under exercise stress. Data from healthy volunteers (n = 15) for cine and real-time free-breathing CMR at rest and under exercise stress were analyzed retrospectively. Exercise stress was performed using an ergometer in the supine position. Segmentations of two deep learning methods, a commercially available technique (comDL) and an openly available network (nnU-Net), were compared to a reference model created via the manual correction of segmentations obtained with comDL. Segmentations of left ventricular endocardium (LV), left ventricular myocardium (MYO), and right ventricle (RV) are compared for both end-systolic and end-diastolic phases and analyzed with Dice’s coefficient. The volumetric analysis includes the cardiac function parameters LV end-diastolic volume (EDV), LV end-systolic volume (ESV), and LV ejection fraction (EF), evaluated with respect to both absolute and relative differences. For cine CMR, nnU-Net and comDL achieve Dice’s coefficients above 0.95 for LV and 0.9 for MYO, and RV. For real-time CMR, the accuracy of nnU-Net exceeds that of comDL overall. For real-time CMR at rest, nnU-Net achieves Dice’s coefficients of 0.94 for LV, 0.89 for MYO, and 0.90 for RV and the mean absolute differences between nnU-Net and the reference are 2.9 mL for EDV, 3.5 mL for ESV, and 2.6% for EF. For real-time CMR under exercise stress, nnU-Net achieves Dice’s coefficients of 0.92 for LV, 0.85 for MYO, and 0.83 for RV and the mean absolute differences between nnU-Net and reference are 11.4 mL for EDV, 2.9 mL for ESV, and 3.6% for EF. Deep learning methods designed or trained for cine CMR segmentation can perform well on real-time CMR. For real-time free-breathing CMR at rest, the performance of deep learning methods is comparable to inter-observer variability in cine CMR and is usable for fully automatic segmentation. For real-time CMR under exercise stress, the performance of nnU-Net could promise a higher degree of automation in the future.

show abstract

How should studies using AI be reported? lessons from a systematic review in cardiac MRI

Cited by 5 publications

References 13 publications

Advancements in cardiac structures segmentation: a comprehensive systematic review of deep learning in CT imaging

Advancements in cardiac structures segmentation: a comprehensive systematic review of deep learning in CT imaging

A systematic review of artificial intelligence tools for chronic pulmonary embolism on CT pulmonary angiography

Assessment of deep learning segmentation for real-time free-breathing cardiac magnetic resonance imaging at rest and under exercise stress

Contact Info

Product

Resources

About