Agreement and Error Rates Using Blinded Review to Evaluate Surgical Pathology of Biopsy Material

Abstract

Blinded review has been shown to be an excellent method to detect disagreements and errors and improve performance in gynecologic cytology. Preliminary studies suggest it may be valuable in surgical pathology. We reviewed 5,000 sequential outpatient surgical pathology biopsy cases without knowledge of the original diagnosis or history and compared the results with those of the original diagnosis. Complete agreement was obtained in 91.12% of cases. The technique of blinded review of surgical pathology biopsy material had a sensitivity of more than 99%, failing to identify an abnormality in 19 cases. Although there was a significant level of diagnostic disagreement (444 cases), primarily due to differences in diagnostic thresholds (292 cases), diagnoses that resulted in a change in the original report (true errors) were present in only 5 cases, and only 4 were clinically significant. This clinically significant error rate of 0.08% is significantly lower than previously published error rates. Blinded review is a sensitive (99%) and effective method to identify areas of disagreement and errors in surgical pathology biopsy material. The relatively high rate of disagreement found with blinded review coupled with the very low rate of error highlights the substantial potential for bias in nonblinded reviews.

A Systematic Review and Meta-analysis of the Diagnostic Accuracy of Frozen Section for Parotid Gland Lesions

“…Threshold effects have been recognized as an important source of variation in other studies. 22 Such variation might be minimized by more uniform application of diagnostic criteria.…”

Increasing Agreement Over Time in Interlaboratory Anatomic Pathology Consultation Material

“…[3][4][5][6][7] Studies using interlaboratory review of pathology material have shown that the rate of disagreement depends on the type of consultation (pathologist directed vs patient/clinician directed 8 ), the source or tissue type of material, 4,[9][10][11] the type of diagnosis, 11 the number of slides examined, 11 the use of immunohistochemistry, 11 and whether the report has more than 1 pathologist's name on it. 11 We recently noted that the rate of disagreement in both our incoming and outgoing patient/clinician-directed interlaboratory anatomic pathology consults had declined over time.…”

When is the practice of pathology malpractice?: Figure 1

“…Within a juridical context, knowledge of the outcome may likewise influence the reviewing pathologist. In this situation in particular, a blinded review is a prerequisite for an impartial evaluation—a problem well known in both pathology and radiology 52 53. On re-examination of missed morphologically determined diagnoses, the type of review can influence the outcome 54–56.…”