Influence of Inter-Annotator Variability on Automatic Mitotic Figure Assessment

Computer-Assisted Mitotic Count Using a Deep Learning-based Algorithm Improves Inter-Observer Reproducibility and Accuracy in canine cutaneous mast cell tumors

“…Even though most participants found the rectangular shape appropriate for the examined ccMCT cases, it might possibly be useful to be able to adjust the MC-ROI shape for tumor types that have a highly variable cellular density and/or may include a variable high proportion of non-neoplastic tissue, that should be excluded from the MC-ROI. 17,29 Even if pathologists examine the same image sections, variability of the number of enumerated MFs has been noted 15,40,41,43 and computer assistance seems to be a promising solution for improving MF identification/classification. Similar to the comparison between stage 2 and 3 of the present study, Pantanowitz et al 30 investigated the influence of computer assistance on the pathologist's performance of annotating MFs in "hpf" images.…”

“…A particular strength of the present study was the use of a pHH3-assisted ground truth for performance evaluation. Most previous studies have evaluated their MF algorithms against a majority vote of pathologists annotations from HE images, 3,5,12,30,35,41,43 which might be problematic due to human limitations in performing this task. If a consensus is generated by such a large group of participants as in the present study, a majority vote is very likely to include predominantly clear MFs and would not contain many morphologically inconclusive or equivocal, but still "true" MFs.…”

“…3,11,42 The pathologist's capability to identify/classify MFs has been evaluated recently. 15,40,41,43 Those studies compared the digital MCs of different pathologists in the same MC-ROIs and revealed an overall difference in number of annotated MFs by a factor of 1.5x to 3.3x. 41,43 This can be attributed to erroneous MF detections related to failure in identification of MF candidates as well as to inaccurate/inconsistent classification of MFs against look-alikes (such as apoptotic bodies, hyperchromatic or deformed nuclei, and inflammatory cells).…”

“…15,40,41,43 Those studies compared the digital MCs of different pathologists in the same MC-ROIs and revealed an overall difference in number of annotated MFs by a factor of 1.5x to 3.3x. 41,43 This can be attributed to erroneous MF detections related to failure in identification of MF candidates as well as to inaccurate/inconsistent classification of MFs against look-alikes (such as apoptotic bodies, hyperchromatic or deformed nuclei, and inflammatory cells). 17 Tabata et al 35 have shown that pathologists have a lower accuracy of MF detection when using WSIs (69-74%) as compared to traditional light microscopy (0.8%).…”

Computer-assisted mitotic count using a deep learning–based algorithm improves interobserver reproducibility and accuracy

“…3,11,46 The pathologist's capability to identify and classify MFs has been evaluated recently. 16,44,45,47 Those studies compared the digital MCs of different pathologists in the same MC-ROIs and revealed an overall difference in number of annotated MFs by a factor of 1.5× to 3.3×. 45,47 This can be attributed to erroneous MF detections related to failure in identification of MF candidates, as well as to inaccurate or inconsistent classification of MFs against lookalikes (such as apoptotic bodies, hyperchromatic or deformed nuclei, and inflammatory cells).…”