A Competition, Benchmark, Code, and Data for Using Artificial Intelligence to Detect Lesions in Digital Breast Tomosynthesis

Abstract

ImportanceAn accurate and robust artificial intelligence (AI) algorithm for detecting cancer in digital breast tomosynthesis (DBT) could significantly improve detection accuracy and reduce health care costs worldwide.ObjectivesTo make training and evaluation data for the development of AI algorithms for DBT analysis available, to develop well-defined benchmarks, and to create publicly available code for existing methods.Design, Setting, and ParticipantsThis diagnostic study is based on a multi-institutional international grand challenge in which research teams developed algorithms to detect lesions in DBT. A data set of 22 032 reconstructed DBT volumes was made available to research teams. Phase 1, in which teams were provided 700 scans from the training set, 120 from the validation set, and 180 from the test set, took place from December 2020 to January 2021, and phase 2, in which teams were given the full data set, took place from May to July 2021.Main Outcomes and MeasuresThe overall performance was evaluated by mean sensitivity for biopsied lesions using only DBT volumes with biopsied lesions; ties were broken by including all DBT volumes.ResultsA total of 8 teams participated in the challenge. The team with the highest mean sensitivity for biopsied lesions was the NYU B-Team, with 0.957 (95% CI, 0.924-0.984), and the second-place team, ZeDuS, had a mean sensitivity of 0.926 (95% CI, 0.881-0.964). When the results were aggregated, the mean sensitivity for all submitted algorithms was 0.879; for only those who participated in phase 2, it was 0.926.Conclusions and RelevanceIn this diagnostic study, an international competition produced algorithms with high sensitivity for using AI to detect lesions on DBT images. A standardized performance benchmark for the detection task using publicly available clinical imaging data was released, with detailed descriptions and analyses of submitted algorithms accompanied by a public release of their predictions and code for selected methods. These resources will serve as a foundation for future research on computer-assisted diagnosis methods for DBT, significantly lowering the barrier of entry for new researchers.

Breast cancer detection with upstream data fusion, machine learning, and automated registration: initial results

“…For radiology, the FROC curve can be based on proximity of a detection to a lesion. For example, this was done in the development of the baseline for the DBTex effort for breast imaging 19 , 25 . A true positive was defined as a detection that satisfies a proximity condition to a lesion ground truth.…”

“…For example, this was done in the development of the baseline for the DBTex effort for breast imaging. 19 , 25 A true positive was defined as a detection that satisfies a proximity condition to a lesion ground truth. Then an FROC curve was defined as the plot of true positive fraction (sensitivity) versus average false positives per DBT volume.…”

“…The DBTex challenge is described in Ref. 25 where participants achieved >90% mean sensitivity at 1, 2, 3, and 4 FPs per volume. The authors state that they believe that one of the areas that should be focused on for future work is incorporating detection information from all available views (CC/MLO), mimicking the way radiologists analyze the images.…”

Lesion localization in digital breast tomosynthesis with deformable transformers by using 2.5D information

“…Dataset: we use the public dataset from Breast Cancer Screening Digital Breast Tomosynthesis (BCS-DBT) Challenge. 6 This dataset has 5060 cases and 22032 DBT scans, including craniocaudal (CC) and mediolateral oblique (MLO) projection. The BCS-DBT dataset comprised four classes, normal, actionable, non biopsy-proven, and biopsy-proven.…”

Sample-efficient framework for breast lesion detection in digital breast tomosynthesis: preliminary analysis on its generalizability

“…The DBTex dataset 2 (phase 1) is a subset of the BCS-DBT dataset, which is publicly available in the Cancer Imaging Archive. 1,2 The DBTex has a total of 1000 breast Tomosynthesis scans from 985 patients with an image dimension of 2457×1996 or 2457×1890, having a DBT volume of around 40-100 slices. Each DBT scan typically consists of 2-4 views, including left/right craniocaudal and mediolateral oblique views.…”