Test-set training improves the detection rates of invasive cancer in screening mammography

Qenam, B.A.; Li, T.; Ekpo, Ernest U.; Frazer, Helen; Brennan, Patrick C.

doi:10.1016/j.crad.2022.11.012

Cited by 3 publications

(7 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The detection rate analyses for the whole study group showed no statistically significant overall change in detection; this may be due to the mix of both the improved and unimproved subgroups, which may have attenuated the power to detect the overall difference in the change. Nonetheless, this finding agrees with the previous studies, which showed that improvements occurred only in the test‐set group and when reading volumes at least met the screening programme requirement of 2000 reads per year 24,26 . However, since the focus of this study was to investigate the test‐set measures for the improved and the unimproved readers, it was essential not to exclude readers based on a factor that influences performance 32,33 …”

Section: Discussionsupporting

confidence: 89%

“…To further investigate the improvements, our team inspected the clinical audit results of test‐set participants, and showed that readers improved significantly in audit metrics 24 including lower rates of recall; higher positive predictive values; higher cancer detection rates. In a subsequent study, focusing on those individuals who read the minimum annual reading numbers (2000 cases) in Australia 25 the performances of test‐set participants improved when being compared with their peers who had no experience of test‐set training 26 . However, it was shown that factors such as reader experience 24 could affect the extent of change.…”

Section: Introductionmentioning

confidence: 98%

“…In a subsequent study, focusing on those individuals who read the minimum annual reading numbers (2000 cases) in Australia 25 the performances of test‐set participants improved when being compared with their peers who had no experience of test‐set training. 26 However, it was shown that factors such as reader experience 24 could affect the extent of change. The current work, focusing on breast cancer detection, aimed to explore performance trends and readers' characteristics that could best describe those who improved most when test‐set training strategies are employed.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Test‐set results can predict participants' development in breast‐screen cancer detection: An observational cohort study

Qenam,

Li,

Alshabibi

et al. 2024

Health Science Reports

View full text Add to dashboard Cite

Background and AimTest‐sets are standardized assessments used to evaluate reader performance in breast screening. Understanding how test‐set results affect real‐world performance can help refine their use as a quality improvement tool. The aim of this study is to explore if mammographic test‐set results could identify breast‐screening readers who improved their cancer detection in association with test‐set training.MethodsTest‐set results of 41 participants were linked to their annual cancer detection rate change in two periods oriented around their first test‐set participation year. Correlation tests and a multiple linear regression model investigated the relationship between each metric in the test‐set results and the change in detection rates. Additionally, participants were divided based on their improvement status between the two periods, and Mann–Whitney U test was used to determine if the subgroups differed in their test‐set metrics.ResultsTest‐set records indicated multiple significant correlations with the change in breast cancer detection rate: a moderate positive correlation with sensitivity (0.688, p < 0.001), a moderate negative correlation with specificity (−0.528, p < 0.001), and a low to moderate positive correlation with lesion sensitivity (0.469, p = 0.002), and the number of years screen‐reading mammograms (0.365, p = 0.02). In addition, the overall regression was statistically significant (F (2,38) = 18.456 p < 0.001), with an R² of 0.493 (adjusted R² = 0.466) based on sensitivity (F = 27.132, p < 0.001) and specificity (F = 9.78, p = 0.003). Subgrouping the cohort based on the change in cancer detection indicated that the improved group is significantly higher in sensitivity (p < 0.001) and lesion sensitivity (p = 0.02) but lower in specificity (p = 0.003).ConclusionSensitivity and specificity are the strongest test‐set performance measures to predict the change in breast cancer detection in real‐world breast screening settings following test‐set participation.

show abstract

Section: Discussionsupporting

confidence: 89%

Section: Introductionmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Test‐set results can predict participants' development in breast‐screen cancer detection: An observational cohort study

Qenam,

Li,

Alshabibi

et al. 2024

Health Science Reports

View full text Add to dashboard Cite

show abstract

“…A dedicated training program with specialized tutorials instructed by breast screening experts and access to self-evaluation mammogram sets with enriched abnormal cases have been suggested to ameliorate the prevalence impact and improve the interpretation skills of local radiologists 39 . The BREAST program has been shown to be an effective training platform for Australian radiologists and radiology trainees in mammographic screening interpretation 20 , 40 , 41 and has demonstrated an improvement in radiologists’ cancer detection in real-world screening programs, 42 , 43 which strongly suggests that the adoption of this type of training could enhance Shanghai radiologists’ skills and improve health outcomes for Chinese women. This has been previously shown by our work in Vietnam, which indicated that the diagnostic performance of local radiologists in detecting small abnormal lesions, such as stellate or spiculated masses, can be increased if a proper training intervention is applied 39 …”

Section: Discussionmentioning

confidence: 99%

Variations of image interpretations of radiologists from different populations in mammography and tomosynthesis with different levels of breast density

Trieu

Xiao

et al. 2023

J. Med. Imag.

View full text Add to dashboard Cite

.PurposeThis study aims to investigate the diagnostic performances of Australian and Shanghai-based Chinese radiologists in reading full-field digital mammogram (FFDM) and digital breast tomosynthesis (DBT) with different levels of breast density.ApproachEighty-two Australian radiologists interpreted a 60-case FFDM set, and 29 radiologists also reported a 35-case DBT set. Sixty Shanghai radiologists read the same FFDM set, and 32 radiologists read the DBT set. The diagnostic performances of Australian and Shanghai radiologists were assessed using truth data (cancer cases were biopsy proven) and compared overall in specificity, case sensitivity, lesion sensitivity, receiver operating characteristics (ROC) area under the curve, and jack-knife free-response receiver operating characteristics (JAFROC) figure of merit, and they were stratified by case characteristics using the Mann–Whitney U test. The Spearman rank test was used to explore the association between radiologists’ performances and their work experience in mammogram interpretation.ResultsThere were significantly higher performances of Australian radiologists compared with Shanghai radiologists in low breast density for case sensitivity, lesion sensitivity, ROC, and JAFROC in the FFDM set (P < 0.0001); in high breast density, Shanghai radiologists’ performances in lesion sensitivity and JAFROC were also lower than Australian radiologists (P < 0.0001). In the DBT test set, Australian radiologists performed better than Shanghai radiologists in cancer detection in both low and high breast density. The work experience of Australian radiologists was positively linked to their diagnostic performances, whereas this association was not statistically significant in Shanghai radiologists.ConclusionThere were significant variations in reading performances between Australian and Shanghai radiologists in FFDM and DBT across different levels of breast density, lesion types, and lesion sizes. An effective training initiative tailored to suit local readers is essential to enhancing the diagnostic accuracy of Shanghai radiologists.

show abstract

“…Self-assessment mammography test sets have been demonstrated to be an effective learning approach in improving the clinical performance of mammography readers, where participants are asked to assess a mammography set enriched with cancer cases, displaying a range of image appearances with known truth [ 1 , 2 ]. Upon completion, participants receive immediate, individual feedback on their performance, including sensitivity and specificity, allowing them to anonymously compare their results with their peers and learn from their mistakes by visualizing their errors ( Figure 1 ).…”

Section: Introductionmentioning

confidence: 99%

Using Radiomics-Based Machine Learning to Create Targeted Test Sets to Improve Specific Mammography Reader Cohort Performance: A Feasibility Study

Tao

Gandomkar

et al. 2023

JPM

View full text Add to dashboard Cite

Mammography interpretation is challenging with high error rates. This study aims to reduce the errors in mammography reading by mapping diagnostic errors against global mammographic characteristics using a radiomics-based machine learning approach. A total of 36 radiologists from cohort A (n = 20) and cohort B (n = 16) read 60 high-density mammographic cases. Radiomic features were extracted from three regions of interest (ROIs), and random forest models were trained to predict diagnostic errors for each cohort. Performance was evaluated using sensitivity, specificity, accuracy, and AUC. The impact of ROI placement and normalization on prediction was investigated. Our approach successfully predicted both the false positive and false negative errors of both cohorts but did not consistently predict location errors. The errors produced by radiologists from cohort B were less predictable compared to those in cohort A. The performance of the models did not show significant improvement after feature normalization, despite the mammograms being produced by different vendors. Our novel radiomics-based machine learning pipeline focusing on global radiomic features could predict false positive and false negative errors. The proposed method can be used to develop group-tailored mammographic educational strategies to help improve future mammography reader performance.

show abstract

Test-set training improves the detection rates of invasive cancer in screening mammography

Cited by 3 publications

References 23 publications

Test‐set results can predict participants' development in breast‐screen cancer detection: An observational cohort study

Test‐set results can predict participants' development in breast‐screen cancer detection: An observational cohort study

Variations of image interpretations of radiologists from different populations in mammography and tomosynthesis with different levels of breast density

Using Radiomics-Based Machine Learning to Create Targeted Test Sets to Improve Specific Mammography Reader Cohort Performance: A Feasibility Study

Contact Info

Product

Resources

About