Going from double to single reading for screening exams labeled as likely normal by AI: what is the impact?

Balta, Christiana; Rodríguez‐Ruiz, Alejandro; Mieskes, Christoph; Karssemeijer, Nico; Heywang‐Köbrunner, Sylvia H.

doi:10.1117/12.2564179

Cited by 18 publications

(32 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The distribution of AI scores is homogeneous for all screening examinations (approximately 10% in each score category), whereas only a minority of screening-detected cancers were scored 1-7: two of 76 DM-based screening-detected cancers (2.6%; 95% CI: 0.72, 9. Given that this group of cases with scores 1-7 includes less than 5% of screening-detected cancers, it was estimated that this is an optimal cutoff point to differentiate likely normal examinations in the proposed AI-based strategies (negative predictive value, 99.98% [95% CI: 99.94, 99.99] in DM and 99.99% [95% CI: 99.95, 99.99] in DBT), similar to previous studies (20).…”

Section: Distribution Of Ai Scoressupporting

confidence: 83%

“…The readers were blinded to the outcomes of the other arms. Some studies have investigated whether AI systems can be used in screening programs to reduce radiologists' workload without negatively affecting the quality of outcomes (18)(19)(20). However, these are limited and have only investigated the use of AI to reduce workload in DM-based screening programs.…”

Section: Original Screening Reading Settingsmentioning

confidence: 99%

“…The AI system used in this study (Transpara, version 1.6.0; Screen-Point Medical) was previously investigated in other publications (17,19,20,(22)(23)(24). This system uses deep learning to detect lesions suspicious for breast cancer on DM and DBT images.…”

Section: Ai Systemmentioning

confidence: 99%

“…In this AI strategy, the least suspicious examinations for AI (those assumed very likely normal with an AI score of 7 or lower, approximately 70% according to the device specifications; the cutoff was chosen based on previous research [20] indicating that replacing double reading with single reading for these very likely normal cases would not reduce screening sensitivity by more than 5%) would not be human-read, and the rest of examinations would be read as in the original setting (single or double reading of DM or DBT images). Additionally, the examinations not recalled by radiologists but within the 2% most suspicious examinations as graded by AI would be automatically recalled in order to potentially improve sensitivity (the cutoff was chosen taking into account radiologists' recall rate at this site).…”

Section: Ai-based Screening Strategymentioning

confidence: 99%

See 3 more Smart Citations

AI-based Strategies to Reduce Workload in Breast Cancer Screening with Mammography and Tomosynthesis: A Retrospective Evaluation

et al. 2021

View full text Add to dashboard Cite

Background:The workflow of breast cancer screening programs could be improved given the high workload and the high number of false-positive and false-negative assessments.Purpose: To evaluate if using an artificial intelligence (AI) system could reduce workload without reducing cancer detection in breast cancer screening with digital mammography (DM) or digital breast tomosynthesis (DBT). Materials and Methods:Consecutive screening-paired and independently read DM and DBT images acquired from January 2015 to December 2016 were retrospectively collected from the Córdoba Tomosynthesis Screening Trial. The original reading settings were single or double reading of DM or DBT images. An AI system computed a cancer risk score for DM and DBT examinations independently. Each original setting was compared with a simulated autonomous AI triaging strategy (the least suspicious examinations for AI are not human-read; the rest are read in the same setting as the original, and examinations not recalled by radiologists but graded as very suspicious by AI are recalled) in terms of workload, sensitivity, and recall rate. The McNemar test with Bonferroni correction was used for statistical analysis.Results: A total of 15 987 DM and DBT examinations (which included 98 screening-detected and 15 interval cancers) from 15 986 women (mean age 6 standard deviation, 58 years 6 6) were evaluated. In comparison with double reading of DBT images (568 hours needed, 92 of 113 cancers detected, 706 recalls in 15 987 examinations), AI with DBT would result in 72.5% less workload (P , .001, 156 hours needed), noninferior sensitivity (95 of 113 cancers detected, P = .38), and 16.7% lower recall rate (P , .001, 588 recalls in 15 987 examinations). Similar results were obtained for AI with DM. In comparison with the original double reading of DM images (222 hours needed, 76 of 113 cancers detected, 807 recalls in 15 987 examinations), AI with DBT would result in 29.7% less workload (P , .001), 25.0% higher sensitivity (P , .001), and 27.1% lower recall rate (P , .001). Conclusion:Digital mammography and digital breast tomosynthesis screening strategies based on artificial intelligence systems could reduce workload up to 70%.

show abstract

Section: Distribution Of Ai Scoressupporting

confidence: 83%

Section: Original Screening Reading Settingsmentioning

confidence: 99%

Section: Ai Systemmentioning

confidence: 99%

Section: Ai-based Screening Strategymentioning

confidence: 99%

See 2 more Smart Citations

AI-based Strategies to Reduce Workload in Breast Cancer Screening with Mammography and Tomosynthesis: A Retrospective Evaluation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…We do not suggest that all screening exams with high AI risk should be recalled, which would result in an unacceptable high recall rate (10%). The cancer frequency in mammography screen exams with risk score 10 is about 44/1000 [30], which means that the majority of the exams are cancer-free. In a prior retrospective study on screening data, we found that the highest proportion of false positives were found in risk group 10, which implies that the mammograms were challenging to analyse both for humans and AI [20].…”

Section: Discussionmentioning

confidence: 99%

Can artificial intelligence reduce the interval cancer rate in mammography screening?

et al. 2021

View full text Add to dashboard Cite

Objectives To investigate whether artificial intelligence (AI) can reduce interval cancer in mammography screening. Materials and methods Preceding screening mammograms of 429 consecutive women diagnosed with interval cancer in Southern Sweden between 2013 and 2017 were analysed with a deep learning–based AI system. The system assigns a risk score from 1 to 10. Two experienced breast radiologists reviewed and classified the cases in consensus as true negative, minimal signs or false negative and assessed whether the AI system correctly localised the cancer. The potential reduction of interval cancer was calculated at different risk score thresholds corresponding to approximately 10%, 4% and 1% recall rates. Results A statistically significant correlation between interval cancer classification groups and AI risk score was observed (p < .0001). AI scored one in three (143/429) interval cancer with risk score 10, of which 67% (96/143) were either classified as minimal signs or false negative. Of these, 58% (83/143) were correctly located by AI, and could therefore potentially be detected at screening with the aid of AI, resulting in a 19.3% (95% CI 15.9–23.4) reduction of interval cancer. At 4% and 1% recall thresholds, the reduction of interval cancer was 11.2% (95% CI 8.5–14.5) and 4.7% (95% CI 3.0–7.1). The corresponding reduction of interval cancer with grave outcome (women who died or with stage IV disease) at risk score 10 was 23% (8/35; 95% CI 12–39). Conclusion The use of AI in screen reading has the potential to reduce the rate of interval cancer without supplementary screening modalities. Key Points • Retrospective study showed that AI detected 19% of interval cancer at the preceding screening exam that in addition showed at least minimal signs of malignancy. Importantly, these were correctly localised by AI, thus obviating supplementary screening modalities. • AI could potentially reduce a proportion of particularly aggressive interval cancers. • There was a correlation between AI risk score and interval cancer classified as true negative, minimal signs or false negative.

show abstract

Artificial intelligence in BreastScreen Norway: a retrospective analysis of a cancer-enriched sample including 1254 breast cancer cases

et al. 2023

View full text Add to dashboard Cite

Objectives To compare results of selected performance measures in mammographic screening for an artificial intelligence (AI) system versus independent double reading by radiologists. Methods In this retrospective study, we analyzed data from 949 screen-detected breast cancers, 305 interval cancers, and 13,646 negative examinations performed in BreastScreen Norway during the period from 2010 to 2018. An AI system scored the examinations from 1 to 10, based on the risk of malignancy. Results from the AI system were compared to screening results after independent double reading. AI score 10 was set as the threshold. The results were stratified by mammographic density. Results A total of 92.7% of the screen-detected and 40.0% of the interval cancers had an AI score of 10. Among women with a negative screening outcome, 9.1% had an AI score of 10. For women with the highest breast density, the AI system scored 100% of the screen-detected cancers and 48.6% of the interval cancers with an AI score of 10, which resulted in a sensitivity of 80.9% for women with the highest breast density for the AI system, compared to 62.8% for independent double reading. For women with screen-detected cancers who had prior mammograms available, 41.9% had an AI score of 10 at the prior screening round. Conclusions The high proportion of cancers with an AI score of 10 indicates a promising performance of the AI system, particularly for women with dense breasts. Results on prior mammograms with AI score 10 illustrate the potential for earlier detection of breast cancers by using AI in screen-reading. Key Points • The AI system scored 93% of the screen-detected cancers and 40% of the interval cancers with AI score 10. • The AI system scored all screen-detected cancers and almost 50% of interval cancers among women with the highest breast density with AI score 10. • About 40% of the screen-detected cancers had an AI score of 10 on the prior mammograms, indicating a potential for earlier detection by using AI in screen-reading.

show abstract

Going from double to single reading for screening exams labeled as likely normal by AI: what is the impact?

Cited by 18 publications

References 0 publications

AI-based Strategies to Reduce Workload in Breast Cancer Screening with Mammography and Tomosynthesis: A Retrospective Evaluation

AI-based Strategies to Reduce Workload in Breast Cancer Screening with Mammography and Tomosynthesis: A Retrospective Evaluation

Can artificial intelligence reduce the interval cancer rate in mammography screening?

Artificial intelligence in BreastScreen Norway: a retrospective analysis of a cancer-enriched sample including 1254 breast cancer cases

Contact Info

Product

Resources

About