Inter-rater agreement in the characterization of cystic renal lesions on contrast-enhanced MRI

Abstract

There is substantial overall inter-observer agreement in the MRI classification of cystic renal lesions. Confidence increases as rater experience increases.

MRI-based Bosniak Classification of Cystic Renal Masses, Version 2019: Interobserver Agreement, Impact of Readers’ Experience, and Diagnostic Performance

“…Bosniak version 2019 had a higher diagnostic specificity for malignancy than did The interobserver agreement in version 2019 was superior to that in previous studies (k = 0.51-0.52) based on MRI (16) or CT (17) scans, probably because of more objective and precise classification criteria. However, agreement was lower than that of other previous studies (k = 0.66-0.85) (10,(18)(19)(20), partially because of the relatively small number of malignant tumors, fewer readers, higher rate of class I and IV CRMs, or the bias in the inclusion of CRMs and establishment of the study cohort in those studies.…”

“…The inclusion of clear-cut Bosniak I and IV CRMs may contribute to higher interobserver agreement (10,21,22). After exclusion of class I and IV CRMs from our study cohort, interobserver agreement was lower with both version 2019 (weighted k = 0.53 vs 0.64) and version 2005 (weighted k = 0.35 vs 0.50), which confirms that Bosniak I and IV CRMs influence interobserver agreement.…”

“…Although it has been applied to MRI, several studies have suggested that MRI can provide more details regarding the wall or septa of CRMs and thus lead to upgrades of CRMs (7)(8)(9). It has also been reported that more experienced senior readers obtain a higher level of agreement than the junior readers when using version 2005 with MRI (10).…”

A practical guide and decision-making protocol for the management of complex renal cystic masses

“…Graumann et al [22] proposed follow-up by US and MRI after initial CT characterisation. The recognised rate of interobserver variation on CT has been shown for MRI as well [14] . MRI has a higher contrast resolution than CT, which could potentially lead to lesion upgrading or downgrading.…”

“…The major known drawback of the BCS is the substantial inter- and intra-observer variation, specifically in the Bosniak IIF and III categories [2] , [3] , [4] . The confidence in agreement is also experience dependent: ‘The more skilled radiologist the less decrease in confidence with regard to CT or MRI’ [4] , [14] . Even in the era of modern imaging, we still perform 40–50% of surgeries for benign lesions in the Bosniak III category [12] , [15] , [16] .…”