The roles of clinical audit and test sets in promoting the quality of breast screening: a scoping review

Qenam, B.A.; Li, Tong; Tapia, Kriscia; Brennan, Patrick C.

doi:10.1016/j.crad.2020.01.015

Cited by 9 publications

(5 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even though the breast imaging services may be distinct in two countries, an effective and efficient training program such as the use of test sets is likely to be beneficial to improve reading skills of radiologists and consequently improve health outcome in the local population. The test‐set‐based educational program, which has been available in Australia for a decade and has been utilised in many other countries, showed on time improvement through providing performance assessment and demonstrating areas of weakness 20 . For instance, the BREAST program is endorsed by RANZCR in Australia as a Continuing Professional Development activity and qualifies as value training to breast screening readers within Australia 9 .…”

Section: Discussionmentioning

confidence: 99%

Differences in lesion interpretation between radiologists in two countries: Lessons from a digital breast tomosynthesis training test set

Gandomkar

Trieu

et al. 2021

Asia-Pac J Clncl Oncology

View full text Add to dashboard Cite

Introduction In many western countries, there is good evidence documenting the performance of radiologists reading digital breast tomosynthesis (DBT) images. However, the diagnostic efficiency of Chinese radiologists using DBT, particularly type of errors being made and type of cancers being missed, is understudied. This study aims to investigate the pattern of diagnostic errors across different lesion types produced by Chinese radiologists diagnosing from DBT images. Australian radiologists will be used as a benchmark. Methods Twelve Chinese radiologists read a DBT test set and located each perceived cancer lesion. True positives, false positives (FP), true negatives and false negatives (FN) were generated. The same test set was also read by 14 Australian radiologists. Z‐scores and Pearson correlations were used to compare interpretation of lesions and identification of normal appearances between two groups of radiologists. Results Architectural distortions (p < .001) and stellate masses (p = .02) were more difficult for Chinese radiologists to correctly diagnose compared to their Australian counterparts. Chinese readers categorised more FPs as discrete masses (p < .001) and fewer FPs as architectural distortions (p < .001) comparing with Australian radiologists. The percentages of FN for each cancer case were not correlated (r = 0.37, p = .18) but the percentages of FP for each normal case were moderately correlated (r = 0.52, p = .02) between two groups of readers. Conclusions Architectural distortions and stellate masses were challenging to Chinese radiologists when reading DBT. Our findings proposed the need of development of training and education programs focussing on imaging cases tailored for specific groups of readers with certain interpretation patterns.

show abstract

Section: Discussionmentioning

confidence: 99%

Differences in lesion interpretation between radiologists in two countries: Lessons from a digital breast tomosynthesis training test set

Gandomkar

Trieu

et al. 2021

Asia-Pac J Clncl Oncology

View full text Add to dashboard Cite

show abstract

“…Clinical audits are the primary source of feedback for breast screening services rather than radiologists, which makes it difficult for radiologists to identify and mitigate their errors. Thus, researchers have identified the need for mammographic test sets as an alternative assessment method with traits that auditing cannot provide, such as expedited and individualized feedback, and precise evaluation in a controlled environment that is based on preconfirmed diagnoses 18 . In 1991, the concept was introduced in the United Kingdom by the PERsonal PerFORmance in Mammographic Screening (PERFORMS) self‐evaluation scheme, 19 which has outlined the test‐set methodology as an educational resource that can provide readers or policymakers with prompt measurement of screen‐reading performance 18 .…”

Section: Introductionmentioning

confidence: 99%

“…Thus, researchers have identified the need for mammographic test sets as an alternative assessment method with traits that auditing cannot provide, such as expedited and individualized feedback, and precise evaluation in a controlled environment that is based on preconfirmed diagnoses. 18 In 1991, the concept was introduced in the United Kingdom by the PERsonal PerFORmance in Mammographic Screening (PERFORMS) self‐evaluation scheme, 19 which has outlined the test‐set methodology as an educational resource that can provide readers or policymakers with prompt measurement of screen‐reading performance. 18 In Australia, test‐set‐based research was established by the Breast Screen Reader Assessment Strategy (BREAST) programme, which introduced digital mammography and automated feedback to the test‐set scheme, 20 thereby enhancing its accessibility and speed.…”

Section: Introductionmentioning

confidence: 99%

“… 18 In 1991, the concept was introduced in the United Kingdom by the PERsonal PerFORmance in Mammographic Screening (PERFORMS) self‐evaluation scheme, 19 which has outlined the test‐set methodology as an educational resource that can provide readers or policymakers with prompt measurement of screen‐reading performance. 18 In Australia, test‐set‐based research was established by the Breast Screen Reader Assessment Strategy (BREAST) programme, which introduced digital mammography and automated feedback to the test‐set scheme, 20 thereby enhancing its accessibility and speed. The DetectEDX programme was also introduced in Australia as a cloud‐based test‐set solution that is more portable.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Test‐set results can predict participants' development in breast‐screen cancer detection: An observational cohort study

Qenam,

Li,

Alshabibi

et al. 2024

Health Science Reports

View full text Add to dashboard Cite

Background and AimTest‐sets are standardized assessments used to evaluate reader performance in breast screening. Understanding how test‐set results affect real‐world performance can help refine their use as a quality improvement tool. The aim of this study is to explore if mammographic test‐set results could identify breast‐screening readers who improved their cancer detection in association with test‐set training.MethodsTest‐set results of 41 participants were linked to their annual cancer detection rate change in two periods oriented around their first test‐set participation year. Correlation tests and a multiple linear regression model investigated the relationship between each metric in the test‐set results and the change in detection rates. Additionally, participants were divided based on their improvement status between the two periods, and Mann–Whitney U test was used to determine if the subgroups differed in their test‐set metrics.ResultsTest‐set records indicated multiple significant correlations with the change in breast cancer detection rate: a moderate positive correlation with sensitivity (0.688, p < 0.001), a moderate negative correlation with specificity (−0.528, p < 0.001), and a low to moderate positive correlation with lesion sensitivity (0.469, p = 0.002), and the number of years screen‐reading mammograms (0.365, p = 0.02). In addition, the overall regression was statistically significant (F (2,38) = 18.456 p < 0.001), with an R² of 0.493 (adjusted R² = 0.466) based on sensitivity (F = 27.132, p < 0.001) and specificity (F = 9.78, p = 0.003). Subgrouping the cohort based on the change in cancer detection indicated that the improved group is significantly higher in sensitivity (p < 0.001) and lesion sensitivity (p = 0.02) but lower in specificity (p = 0.003).ConclusionSensitivity and specificity are the strongest test‐set performance measures to predict the change in breast cancer detection in real‐world breast screening settings following test‐set participation.

show abstract

“…This exposure to a wide range of mammographic appearances, the high prevalence of cancer cases, and the immediate feedback with visual display make self-assessment mammography test sets an effective approach for developing radiologists’ expertise [ 3 ]. However, most self-assessment modules available use a one-size-fits-all approach to selecting training cases without considering the particular needs of specific cohorts of radiologists [ 4 ]. An alternative solution for customizing the test sets used would be cohort-tailored self-assessment test sets, with an emphasis on mammographic appearances that are particularly challenging for that group [ 5 ].…”

Section: Introductionmentioning

confidence: 99%

Using Radiomics-Based Machine Learning to Create Targeted Test Sets to Improve Specific Mammography Reader Cohort Performance: A Feasibility Study

Tao

Gandomkar

et al. 2023

JPM

View full text Add to dashboard Cite

Mammography interpretation is challenging with high error rates. This study aims to reduce the errors in mammography reading by mapping diagnostic errors against global mammographic characteristics using a radiomics-based machine learning approach. A total of 36 radiologists from cohort A (n = 20) and cohort B (n = 16) read 60 high-density mammographic cases. Radiomic features were extracted from three regions of interest (ROIs), and random forest models were trained to predict diagnostic errors for each cohort. Performance was evaluated using sensitivity, specificity, accuracy, and AUC. The impact of ROI placement and normalization on prediction was investigated. Our approach successfully predicted both the false positive and false negative errors of both cohorts but did not consistently predict location errors. The errors produced by radiologists from cohort B were less predictable compared to those in cohort A. The performance of the models did not show significant improvement after feature normalization, despite the mammograms being produced by different vendors. Our novel radiomics-based machine learning pipeline focusing on global radiomic features could predict false positive and false negative errors. The proposed method can be used to develop group-tailored mammographic educational strategies to help improve future mammography reader performance.

show abstract

The roles of clinical audit and test sets in promoting the quality of breast screening: a scoping review

Cited by 9 publications

References 51 publications

Differences in lesion interpretation between radiologists in two countries: Lessons from a digital breast tomosynthesis training test set

Differences in lesion interpretation between radiologists in two countries: Lessons from a digital breast tomosynthesis training test set

Test‐set results can predict participants' development in breast‐screen cancer detection: An observational cohort study

Using Radiomics-Based Machine Learning to Create Targeted Test Sets to Improve Specific Mammography Reader Cohort Performance: A Feasibility Study

Contact Info

Product

Resources

About