Optimizing the Pairs of Radiologists That Double Read Screening                     Mammograms

Gommers, Jessie J. J.; Abbey, Craig K.; Strand, Fredrik; Taylor-Phillips, Sian; Jenkinson, David J.; Larsen, Marthe; Hofvind, Solveig; Sechopoulos, Ioannis; Broeders, Mireille J. M.

doi:10.1148/radiol.222691

Cited by 3 publications

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

A bivariate binormal model for modelling double reading of screening mammograms

Abbey,

Gommers,

Eckstein

et al. 2024

Medical Imaging 2024: Image Perception, Observer Performance, and Technology Assessment

Self Cite

View full text Add to dashboard Cite

Double reading of screening mammograms, a feature of many breast cancer screening programs, is impacted by interactions between the two image readers. In this work, we describe how the bivariate binormal (BVBN) model, originally developed for statistical analysis of reader studies, can be used to analyze double reading of screening mammograms. The model posits two bivariate normal distributions that describe the distribution of latent decision variables of the two readers for cancer and non-cancer cases. The BVBN allows for the estimation of correlation coefficients between the decision variables of two readers, independent of performance and the threshold for recall. We contend that these correlation coefficients are a useful way to characterize interactions between readers because they characterize associations at the level of the perceptual response in a way that is consistent with Signal Detection Theory. We describe the BVBN model and show how parameters can be estimated from count data under an assumed multinomial distribution.The analysis presented focuses on two aspects of the BVBN model. For implementation using binary data, an equal-variance assumption on latent decision variables is required. Otherwise, the model is over-parameterized. We characterize and discuss the consequence of this assumption. We also show how disagreement rates, an alternative measure of reader interactions, suffer from base-rate effects making them more difficult to interpret than the correlation coefficients of the BVBN model.

show abstract

A bivariate binormal model for modelling double reading of screening mammograms

Abbey,

Gommers,

Eckstein

et al. 2024

Medical Imaging 2024: Image Perception, Observer Performance, and Technology Assessment

Self Cite

View full text Add to dashboard Cite

show abstract

AI as a Secondary Reader in Mammography

Batan

2024

Radiology

View full text Add to dashboard Cite

Modeling Radiologists’ Assessments to Explore Pairing Strategies for Optimized Double Reading of Screening Mammograms

Gommers,

Abbey,

Strand

et al. 2024

Med Decis Making

View full text Add to dashboard Cite

Purpose To develop a model that simulates radiologist assessments and use it to explore whether pairing readers based on their individual performance characteristics could optimize screening performance. Methods Logistic regression models were designed and used to model individual radiologist assessments. For model evaluation, model-predicted individual performance metrics and paired disagreement rates were compared against the observed data using Pearson correlation coefficients. The logistic regression models were subsequently used to simulate different screening programs with reader pairing based on individual true-positive rates (TPR) and/or false-positive rates (FPR). For this, retrospective results from breast cancer screening programs employing double reading in Sweden, England, and Norway were used. Outcomes of random pairing were compared against those composed of readers with similar and opposite TPRs/FPRs, with positive assessments defined by either reader flagging an examination as abnormal. Results The analysis data sets consisted of 936,621 (Sweden), 435,281 (England), and 1,820,053 (Norway) examinations. There was good agreement between the model-predicted and observed radiologists’ TPR and FPR ( r ≥ 0.969). Model-predicted negative-case disagreement rates showed high correlations ( r ≥ 0.709), whereas positive-case disagreement rates had lower correlation levels due to sparse data ( r ≥ 0.532). Pairing radiologists with similar FPR characteristics (Sweden: 4.50% [95% confidence interval: 4.46%–4.54%], England: 5.51% [5.47%–5.56%], Norway: 8.03% [7.99%–8.07%]) resulted in significantly lower FPR than with random pairing (Sweden: 4.74% [4.70%–4.78%], England: 5.76% [5.71%–5.80%], Norway: 8.30% [8.26%–8.34%]), reducing examinations sent to consensus/arbitration while the TPR did not change significantly. Other pairing strategies resulted in equal or worse performance than random pairing. Conclusions Logistic regression models accurately predicted screening mammography assessments and helped explore different radiologist pairing strategies. Pairing readers with similar modeled FPR characteristics reduced the number of examinations unnecessarily sent to consensus/arbitration without significantly compromising the TPR. Highlights A logistic-regression model can be derived that accurately predicts individual and paired reader performance during mammography screening reading. Pairing screening mammography radiologists with similar false-positive characteristics reduced false-positive rates with no significant loss in true positives and may reduce the number of examinations unnecessarily sent to consensus/arbitration.

show abstract

Optimizing the Pairs of Radiologists That Double Read Screening Mammograms

Cited by 3 publications

References 26 publications

A bivariate binormal model for modelling double reading of screening mammograms

A bivariate binormal model for modelling double reading of screening mammograms

AI as a Secondary Reader in Mammography

Modeling Radiologists’ Assessments to Explore Pairing Strategies for Optimized Double Reading of Screening Mammograms

Contact Info

Product

Resources

About