Observer Variability and the Performance between Faculties and Residents: US Criteria for Benign and Malignant Thyroid Nodules

Abstract

ObjectiveTo evaluate the interobserver variability and performance in the interpretation of ultrasonographic (US) findings of thyroid nodules.Materials and Methods72 malignant nodules and 61 benign nodules were enrolled as part of this study. Five faculty radiologists and four residents independently performed a retrospective analysis of the US images. The observers received one training session after the first interpretation and then performed a secondary interpretation. Agreement was analyzed by Cohen's kappa statistic. Degree of performance was analyzed using receiver operating characteristic (ROC) curves.ResultsAgreement between the faculties was fair-to-good for all criteria; however, between residents, agreement was poor-to-fair. The area under the ROC curves was 0.72, 0.62, and 0.60 for the faculties, senior residents, and junior residents, respectively. There was a significant difference in performance between the faculties and the residents (p < 0.05). There was a significant increase in the agreement for some criteria in the faculties and the senior residents after the training session, but no significant increase in the junior residents.ConclusionIndependent reporting of thyroid US performed by residents is undesirable. A continuous and specialized resident training is essential to enhance the degree of agreement and performance.

Application of Thyroid Imaging Reporting and Data System in the Ultrasound Assessment of Thyroid Nodules According to Physician Experience

“…8,[22][23][24][25] The interobserver agreement of the final assessment of thyroid nodules has been reported to be substantial among experienced radiologists (κ = 0.72-0.79), although the κ values have been reported to range from slight to substantial in individual US feature assessment. 23 Furthermore, the interobserver agreement among faculty staff was greater than among residents (κ = 0.55 and 0.11, respectively; P < 0.05), 22 In addition, the diagnostic performance of less experienced physicians was significantly lower (AUC, 0.60-0.62) than that of faculty physicians (AUC, 0.72). 23,24 Therefore, experience can be a critical factor affecting the diagnostic accuracy for thyroid nodules in the application of TIRADS.…”

All Thyroid Ultrasound Evaluations are Not Equal: Sonographers Specialized in Thyroid Cancer Correctly Label Clinical N0 Disease in Well Differentiated Thyroid Cancer

“…US is a highly operator dependent modality, and variability in image interpretation between sonographers is problematic 15, 16, 24, 25, 29 . Rosario evaluated US assessment of the cervical LN during surveillance in patients with known high risk PTC 29 .…”

“…Radiology educational literature emphasizes the importance of repetition, in addition to familiarity with the key imaging characteristics, for greater accuracy of thyroid US interpretation 24, 25 . Thyroid surgeons, by using US weekly in both the clinic and operating room, can quickly develop the skills needed to proficiently and accurately perform thyroid US.…”

“…Use of US during surgical training has become integrated into multiple different specialties; trauma, breast, abdominal, vascular, critical care, and head and neck surgery 23 . As interpretation of US images can vary greatly, expertise in thyroid imaging as well as consistency of whom is performing the study results in optimal outcomes 11, 15, 16, 24, 25 . Access to a specialized thyroid sonographer is not available at all institutions.…”

Off-site evaluation of three-dimensional ultrasound for the diagnosis of thyroid nodules: comparison with two-dimensional ultrasound

“…However, with respect to diagnostic accuracy, current 2-dimensional (2D) US has been reported to show limited accuracy in the prediction of malignancy in thyroid nodules and the prediction of extrathyroidal extension in malignant thyroid nodules [3][4][5]. In addition, 2D US for the thyroid gland has been reported to be also limited in terms of the considerable interobserver variability, as is the case with other organs [6][7][8].…”