Validation of digital microscopy: Review of validation methods and sources of bias

Abstract

Digital microscopy (DM) is increasingly replacing traditional light microscopy (LM) for performing routine diagnostic and research work in human and veterinary pathology. The DM workflow encompasses specimen preparation, whole-slide image acquisition, slide retrieval, and the workstation, each of which has the potential (depending on the technical parameters) to introduce limitations and artifacts into microscopic examination by pathologists. Performing validation studies according to guidelines established in human pathology ensures that the best-practice approaches for patient care are not deteriorated by implementing DM. Whereas current publications on validation studies suggest an overall high reliability of DM, each laboratory is encouraged to perform an individual validation study to ensure that the DM workflow performs as expected in the respective clinical or research environment. With the exception of validation guidelines developed by the College of American Pathologists in 2013 and its update in 2021, there is no current review of the application of methods fundamental to validation. We highlight that there is high methodological variation between published validation studies, each having advantages and limitations. The diagnostic concordance rate between DM and LM is the most relevant outcome measure, which is influenced (regardless of the viewing modality used) by different sources of bias including complexity of the cases examined, diagnostic experience of the study pathologists, and case recall. Here, we review 3 general study designs used for previous publications on DM validation as well as different approaches for avoiding bias.

Standardisation of canine meningioma grading: Inter‐observer agreement and recommendations for reproducible histopathologic criteria

“…27,29 For all the evaluated criteria, intra-observer agreement was substantial to almost perfect showing that the disagreement was almost all due to inter-observer disagreement. 35 As in human neuropathology, a regular discussion on different cases should be encouraged in veterinary pathology, to better define the more subjective features of a disease entity, to homogenously educate pathologists and, ultimately, to increase diagnostic accuracy. 17,24,30 Robust and reproducible histologic tumour diagnosis and grading is an essential part of providing adequate patient care and treatment.…”

“…The use of digitalized slides probably contributed to the low inter-observer agreement, because of the lack of validated process, the lack of users training on digital pathology and the lack of standardised workstation. 35 It is out of the scope of the present manuscript to give recommendations on digital pathology, but readers are strongly encouraged to read the most recent specialised literature on the topic.…”

Cytologic scoring of equine exercise-induced pulmonary hemorrhage: Performance of human experts and a deep learning-based algorithm

“…The use of digital microscopy for cytological specimens is, however, hampered by limited image resolution and lack of fine focus of default WSIs. 7 , 8 Nevertheless, the annotators of the present study consider WSIs appropriate to perform the THS in this study’s cases, because of the uniform depth of the samples, and as relatively little cellular detail is necessary to evaluate the intracytoplasmic hemosiderin content of macrophages. Another limitation of WSIs specific to the task of scoring hemosiderophages is that different WSI scanners often exhibit a marked difference in the color representation, ie, they might have a higher or lower intensity of the blue color.…”

Canine Vaginal Cytology: A Revised Definition of Exfoliated Vaginal Cells

“…There is a strong public interest and market forces that are driving the rapid development of such diagnostic procedures ( 34 ). Also in challenging on-site staffing situations, such as the COVID 19 pandemic, digital microscopy may be an important tool to keep histology workflows running smoothly ( 35 ). Some studies have shown that AI is even partially superior to human experts in cytology, namely in determining neoplastic vs. normal cells ( 36 ).…”