Interobserver reproducibility of The Paris System for Reporting Urinary Cytology

Abstract

Background:The Paris System for Reporting Urinary Cytology represents a significant improvement in classification of urinary specimens. The system acknowledges the difficulty in cytologically diagnosing low-grade urothelial carcinomas and has developed categories to deal with this issue. The system uses six categories: unsatisfactory, negative for high-grade urothelial carcinoma (NHGUC), atypical urothelial cells, suspicious for high-grade urothelial carcinoma, high-grade urothelial carcinoma, other malignancies and a seventh subcategory (low-grade urothelial neoplasm).Methods:Three hundred and fifty-seven urine specimens were independently reviewed by four cytopathologists unaware of the previous diagnoses. Each cytopathologist rendered a diagnosis according to the Paris System categories. Agreement was assessed using absolute agreement and weighted chance-corrected agreement (kappa). Disagreements were classified as low impact and high impact based on the potential impact of a misclassification on clinical management.Results:The average absolute agreement was 65% with an average expected agreement of 44%. The average chance-corrected agreement (kappa) was 0.32. Nine hundred and ninety-nine of 1902 comparisons between rater pairs were in agreement, but 12% of comparisons differed by two or more categories for the category NHGUC. Approximately 15% of the disagreements were classified as high clinical impact.Conclusions:Our findings indicated that the scheme recommended by the Paris System shows adequate precision for the category NHGUC, but the other categories demonstrated unacceptable interobserver variability. This low level of diagnostic precision may negatively impact the applicability of the Paris System for widespread clinical application.

The Milan system for reporting salivary gland cytopathology: A study of inter‐observer reproducibility

“…In a study reporting inter‐observer reproducibility for the Paris system of reporting urinary cytology, the authors found an average absolute agreement of 65% with an average expected agreement of 44%. The calculated Kappa was 0.32 . Importantly, approximately 15% of disagreements were classified as having a high clinical impact.…”

“…For a diagnostic categorization system to be clinically useful it must show good inter‐observer reproducibility. Previously published diagnostic categorization schemes for urinary cytology and respiratory cytology have had variable degrees of inter‐observer reproducibility . We are unaware of a prior study investigating the inter‐observer diagnostic reproducibility of the Milan System for Salivary Gland Cytopathology.…”

Practice Patterns in Urinary Cytopathology Prior to the Paris System for Reporting Urinary Cytology

“…Third, because the questionnaire was mailed in July 2014, the data of TPS utilization may not reflect the current situation, at least in certain aspects, because TPS was published in 2016, and since then many institutions have likely implemented it. [7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26] Finally, although information about TPS has been widely disseminated, in particular in the cytopathology community, there is still a need for education about TPS, most importantly for our clinical colleagues.…”

The Paris system of Reporting Urinary Cytology: Strengths and opportunities

“…Studies have shown that using the set of standardized criteria in TPS has generally resulted in a reduction of the interinstitutional variation of “atypical urothelial cells” (AUC) rates compared with their pre‐Paris rates, from 6.60% to 41.20% (a 34.6% range) to 5.30% to 26.00% (a 20.7% range) 12‐18 . Most institutions experienced decreases in AUC rates; the sole outlier in a recent review of the literature was an institution with very low pre‐Paris AUC rates and the reason for an increase in AUC in this single institution was an overinterpretation of the morphologic criteria for atypia 19 .…”